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GENERAL PYTHON FAQ 


1.1 General Information 


1.1.1 What is Python? 


Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, 
dynamic typing, very high level dynamic data types, and classes. It supports multiple programming paradigms beyond 
object-oriented programming, such as procedural and functional programming. Python combines remarkable power 
with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, 
and is extensible in C or C++. It is also usable as an extension language for applications that need a programmable 
interface. Finally, Python is portable: it runs on many Unix variants including Linux and macOS, and on Windows. 


To find out more, start with tutorial-index. The Beginner's Guide to Python links to other introductory tutorials and 
resources for learning Python. 


1.1.2 What is the Python Software Foundation? 


The Python Software Foundation is an independent non-profit organization that holds the copyright on Python versions 
2.1 and newer. The PSF’s mission is to advance open source technology related to the Python programming language 
and to publicize the use of Python. The PSF’s home page is at https://www.python.org/psf/. 


Donations to the PSF are tax-exempt in the US. If you use Python and find it helpful, please contribute via the PSF 
donation page. 


1.1.3 Are there copyright restrictions on the use of Python? 


You can do anything you want with the source, as long as you leave the copyrights in and display those copyrights 
in any documentation about Python that you produce. If you honor the copyright rules, its OK to use Python for 
commercial use, to sell copies of Python in source or binary form (modified or unmodified), or to sell products that 
incorporate Python in some form. We would still like to know about all commercial use of Python, of course. 


See the PSF license page to find further explanations and a link to the full text of the license. 


The Python logo is trademarked, and in certain cases permission is required to use it. Consult the Trademark Usage 
Policy for more information. 
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1.1.4 Why was Python created in the first place? 


Here's a very brief summary of what started it all, written by Guido van Rossum: 


I had extensive experience with implementing an interpreted language in the ABC group at CWI, and 
from working with this group I had learned a lot about language design. This is the origin of many Python 
features, including the use of indentation for statement grouping and the inclusion of very-high-level data 
types (although the details are all different in Python). 


I had a number of gripes about the ABC language, but also liked many of its features. It was impossi- 
ble to extend the ABC language (or its implementation) to remedy my complaints — in fact its lack of 
extensibility was one of its biggest problems. I had some experience with using Modula-2+ and talked 
with the designers of Modula-3 and read the Modula-3 report. Modula-3 is the origin of the syntax and 
semantics used for exceptions, and some other Python features. 


I was working in the Amoeba distributed operating system group at CWI. We needed a better way to 
do system administration than by writing either C programs or Bourne shell scripts, since Amoeba had 
its own system call interface which wasn’t easily accessible from the Bourne shell. My experience with 
error handling in Amoeba made me acutely aware of the importance of exceptions as a programming 
language feature. 


It occurred to me that a scripting language with a syntax like ABC but with access to the Amoeba system 
calls would fill the need. I realized that it would be foolish to write an Amoeba-specific language, so I 
decided that I needed a language that was generally extensible. 


During the 1989 Christmas holidays, I had a lot of time on my hand, so I decided to give it a try. During 
the next year, while still mostly working on it in my own time, Python was used in the Amoeba project 
with increasing success, and the feedback from colleagues made me add many early improvements. 


In February 1991, after just over a year of development, I decided to post to USENET. The rest is in 
the Misc/HISTORY file. 


1.1.5 What is Python good for? 


Python is a high-level general-purpose programming language that can be applied to many different classes of prob- 
lems. 


The language comes with a large standard library that covers areas such as string processing (regular expressions, 
Unicode, calculating differences between files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP), 
software engineering (unit testing, logging, profiling, parsing Python code), and operating system interfaces (system 
calls, filesystems, TCP/IP sockets). Look at the table of contents for library-index to get an idea of what's available. 
A wide variety of third-party extensions are also available. Consult the Python Package Index to find packages of 
interest to you. 


1.1.6 How does the Python version numbering scheme work? 


Python versions are numbered “A.B.C” or “A.B”: 
e A is the major version number — it is only incremented for really major changes in the language. 
e Bis the minor version number — it is incremented for less earth-shattering changes. 
e C is the micro version number — it is incremented for each bugfix release. 

See PEP 6 for more information about bugfix releases. 


Not all releases are bugfix releases. In the run-up to a new major release, a series of development releases are made, 
denoted as alpha, beta, or release candidate. Alphas are early releases in which interfaces aren’t yet finalized; it’s 
not unexpected to see an interface change between two alpha releases. Betas are more stable, preserving existing 
interfaces but possibly adding new modules, and release candidates are frozen, making no changes except as needed 
to fix critical bugs. 


Alpha, beta and release candidate versions have an additional suffix: 
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e The suffix for an alpha version is “aN” for some small number N. 
+ The suffix for a beta version is “bN” for some small number N. 
+ The suffix for a release candidate version is “rcN” for some small number N. 


In other words, all versions labeled 2.0aN precede the versions labeled 2.0bN, which precede versions labeled 2.0rcN, 
and those precede 2.0. 


You may also find version numbers with a “+” suffix, e.g. “2.24”. These are unreleased versions, built directly from 
the CPython development repository. In practice, after a final minor release is made, the version is incremented to 
the next minor version, which becomes the “a0” version, e.g. “2.4a0”. 


See also the documentation for sys. version, sys.hexversion, and sys.version_info. 


1.1.7 How do | obtain a copy of the Python source? 
The latest Python source distribution is always available from python.org, at https://www.python.org/downloads/. 
The latest development sources can be obtained at https://github.com/python/cpython/. 


The source distribution is a gzipped tar file containing the complete C source, Sphinx-formatted documentation, 
Python library modules, example programs, and several useful pieces of freely distributable software. The source 
will compile and run out of the box on most UNIX platforms. 


Consult the Getting Started section of the Python Developer's Guide for more information on getting the source code 
and compiling it. 


1.1.8 How do | get documentation on Python? 
The standard documentation for the current stable version of Python is available at https://docs.python.org/3/. PDF, 
plain text, and downloadable HTML versions are also available at https://docs.python.org/3/download.html. 


The documentation is written in reStructuredText and processed by the Sphinx documentation tool. The reStruc- 
turedText source for the documentation is part of the Python source distribution. 


1.1.9 I’ve never programmed before. Is there a Python tutorial? 


There are numerous tutorials and books available. The standard documentation includes tutorial-index. 


Consult the Beginner’s Guide to find information for beginning Python programmers, including lists of tutorials. 


1.1.10 Is there a newsgroup or mailing list devoted to Python? 


There is a newsgroup, comp. lang. python, and a mailing list, python-list. The newsgroup and mailing list are 
gatewayed into each other — if you can read news it’s unnecessary to subscribe to the mailing list. comp. lang. 
python is high-traffic, receiving hundreds of postings every day, and Usenet readers are often more able to cope 
with this volume. 


Announcements of new software releases and events can be found in comp.lang.python.announce, a low-traffic mod- 
erated list that receives about five postings per day. It’s available as the python-announce mailing list. 


More info about other mailing lists and newsgroups can be found at https://www.python.org/community/lists/. 
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1.1.11 How do | get a beta test version of Python? 
Alpha and beta releases are available from https://www.python.org/downloads/. All releases are announced on the 


comp.lang.python and comp.lang.python.announce newsgroups and on the Python home page at https://www.python. 
org/; an RSS feed of news is available. 


You can also access the development version of Python through Git. See The Python Developer's Guide for details. 


1.1.12 How do | submit bug reports and patches for Python? 


To report a bug or submit a patch, use the issue tracker at https://github.com/python/cpython/issues. 


For more information on how Python is developed, consult the Python Developer's Guide. 


1.1.13 Are there any published articles about Python that | can reference”? 


It’s probably best to cite your favorite book about Python. 
The very first article about Python was written in 1991 and is now quite outdated. 


Guido van Rossum and Jelke de Boer, “Interactively Testing Remote Servers Using the Python Pro- 
gramming Language”, CWI Quarterly, Volume 4, Issue 4 (December 1991), Amsterdam, pp 283-303. 


1.1.14 Are there any books on Python? 


Yes, there are many, and more are being published. See the python.org wiki at https://wiki.python.org/moin/ 
PythonBooks for a list. 


You can also search online bookstores for “Python” and filter out the Monty Python references; or perhaps search for 
“Python” and “language”. 


1.1.15 Where in the world is www.python.org located? 


The Python project’s infrastructure is located all over the world and is managed by the Python Infrastructure Team. 
Details here. 


1.1.16 Why is it called Python? 
When he began implementing Python, Guido van Rossum was also reading the published scripts from “Monty 


Python’s Flying Circus”, a BBC comedy series from the 1970s. Van Rossum thought he needed a name that was 
short, unique, and slightly mysterious, so he decided to call the language Python. 


1.1.17 Do I have to like “Monty Python’s Flying Circus”? 


No, but it helps. :) 
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1.2 Python in the real world 


1.2.1 How stable is Python? 


Very stable. New, stable releases have been coming out roughly every 6 to 18 months since 1991, and this seems 
likely to continue. As of version 3.9, Python will have a major new release every 12 months (PEP 602). 


The developers issue “bugfix” releases of older versions, so the stability of existing releases gradually improves. Bugfix 
releases, indicated by a third component of the version number (e.g. 3.5.3, 3.6.2), are managed for stability; only 
fixes for known problems are included in a bugfix release, and it’s guaranteed that interfaces will remain the same 
throughout a series of bugfix releases. 


The latest stable releases can always be found on the Python download page. There are two production-ready versions 
of Python: 2.x and 3.x. The recommended version is 3.x, which is supported by most widely used libraries. Although 
2.x is still widely used, it is not maintained anymore. 


1.2.2 How many people are using Python? 


There are probably millions of users, though it’s difficult to obtain an exact count. 


Python is available for free download, so there are no sales figures, and it’s available from many different sites and 
packaged with many Linux distributions, so download statistics don’t tell the whole story either. 


The comp.lang.python newsgroup is very active, but not all Python users post to the group or even read it. 


1.2.3 Have any significant projects been done in Python? 


See https://www.python.org/about/success for a list of projects that use Python. Consulting the proceedings for past 
Python conferences will reveal contributions from many different companies and organizations. 


High-profile Python projects include the Mailman mailing list manager and the Zope application server. Several Linux 
distributions, most notably Red Hat, have written part or all of their installer and system administration software in 
Python. Companies that use Python internally include Google, Yahoo, and Lucasfilm Ltd. 


1.2.4 What new developments are expected for Python in the future? 


See https://peps.python.org/ for the Python Enhancement Proposals (PEPs). PEPs are design documents describing 
a suggested new feature for Python, providing a concise technical specification and a rationale. Look for a PEP titled 
“Python X.Y Release Schedule”, where X.Y is a version that hasn’t been publicly released yet. 


New development is discussed on the python-dev mailing list. 


1.2.5 Is it reasonable to propose incompatible changes to Python? 


In general, no. There are already millions of lines of Python code around the world, so any change in the language 
that invalidates more than a very small fraction of existing programs has to be frowned upon. Even if you can provide 
a conversion program, there’s still the problem of updating all documentation; many books have been written about 
Python, and we don’t want to invalidate them all at a single stroke. 


Providing a gradual upgrade path is necessary if a feature has to be changed. PEP 5 describes the procedure followed 
for introducing backward-incompatible changes while minimizing disruption for users. 
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1.2.6 Is Python a good language for beginning programmers? 


Yes. 


It is still common to start students with a procedural and statically typed language such as Pascal, C, or a subset of 
C++ or Java. Students may be better served by learning Python as their first language. Python has a very simple and 
consistent syntax and a large standard library and, most importantly, using Python in a beginning programming course 
lets students concentrate on important programming skills such as problem decomposition and data type design. With 
Python, students can be quickly introduced to basic concepts such as loops and procedures. They can probably even 
work with user-defined objects in their very first course. 


For a student who has never programmed before, using a statically typed language seems unnatural. It presents 
additional complexity that the student must master and slows the pace of the course. The students are trying to learn 
to think like a computer, decompose problems, design consistent interfaces, and encapsulate data. While learning 
to use a statically typed language is important in the long term, it is not necessarily the best topic to address in the 
students’ first programming course. 


Many other aspects of Python make it a good first language. Like Java, Python has a large standard library so 
that students can be assigned programming projects very early in the course that do something. Assignments aren’t 
restricted to the standard four-function calculator and check balancing programs. By using the standard library, 
students can gain the satisfaction of working on realistic applications as they learn the fundamentals of programming. 
Using the standard library also teaches students about code reuse. Third-party modules such as PyGame are also 
helpful in extending the students’ reach. 


Python’s interactive interpreter enables students to test language features while they’re programming. They can keep 
a window with the interpreter running while they enter their program’s source in another window. If they can’t 
remember the methods for a list, they can do something like this: 


>>> L = [] 

>>> dir (L) 

["_ add__', ' class ES contains__', '_ delattr__ ', '_ delitem_ ', 
rt are "y t dee ty, > eq__', ' format__', '__ge a 

'_ getattribute__', '__getitem__', '__gt ts t hash _*, > iadd__', 
imul T3 | init Ya ! iter_', ' le Ta! len__', ' LE F 

r mul ', '_ ne r *_hew_*, ? reduce Tp! reduce_ex_ ', 

t repr Tgp! reversed__', ' rmul ra t setattr__', '_ setitem_ ', 
tt. sizeoE "ps? STE A subclasshook__', 'append', 'clear', 


'"copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 

'reverse', 'sort'] 

>>> [d for d in dir(L) 1£ "__" not an d] 

['append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 
>'reverse', 'sort'] 


>>> help (L.append) 
Help on built-in function append: 


append(...) 
L.append (object) -> None -- append object to end 


>>> L.append (1) 
>>> L 


With the interpreter, documentation is never far from the student as they are programming. 


There are also good IDEs for Python. IDLE is a cross-platform IDE for Python that is written in Python using 
Tkinter. Emacs users will be happy to know that there is a very good Python mode for Emacs. All of these pro- 
gramming environments provide syntax highlighting, auto-indenting, and access to the interactive interpreter while 
coding. Consult the Python wiki for a full list of Python editing environments. 


If you want to discuss Python’s use in education, you may be interested in joining the edu-sig mailing list. 
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CHAPTER 
TWO 


PROGRAMMING FAQ 


2.1 General Questions 


2.1.1 Is there a source code level debugger with breakpoints, single-stepping, 
etc.? 


Yes. 


Several debuggers for Python are described below, and the built-in function breakpoint () allows you to drop 
into any of them. 


The pdb module is a simple but adequate console-mode debugger for Python. It is part of the standard Python library, 
and is documented in the Library Reference Manual. You can also write your own debugger by 
using the code for pdb as an example. 


The IDLE interactive development environment, which is part of the standard Python distribution (normally available 
as Tools/scripts/idle3), includes a graphical debugger. 


Python Win is a Python IDE that includes a GUI debugger based on pdb. The Python Win debugger colors breakpoints 
and has quite a few cool features such as debugging non-PythonWin programs. PythonWin is available as part of 
pywin32 project and as a part of the ActivePython distribution. 


Eric is an IDE built on PyQt and the Scintilla editing component. 
trepan3k is a gdb-like debugger. 
Visual Studio Code is an IDE with debugging tools that integrates with version-control software. 
There are a number of commercial Python IDEs that include graphical debuggers. They include: 
e Wing IDE 
e Komodo IDE 
e PyCharm 


2.1.2 Are there tools to help find bugs or perform static analysis? 


Yes. 
Pylint and Pyflakes do basic checking that will help you catch bugs sooner. 


Static type checkers such as Mypy, Pyre, and Pytype can check type hints in Python source code. 
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2.1.3 How can | create a stand-alone binary from a Python script? 


You don't need the ability to compile Python to C code if all you want is a stand-alone program that users can 
download and run without having to install the Python distribution first. There are a number of tools that determine 
the set of modules required by a program and bind these modules together with a Python binary to produce a single 
executable. 


One is to use the freeze tool, which is included in the Python source tree as Tools/freeze. It converts Python byte 
code to C arrays; with a C compiler you can embed all your modules into a new program, which is then linked with 
the standard Python modules. 


It works by scanning your source recursively for import statements (in both forms) and looking for the modules in the 
standard Python path as well as in the source directory (for built-in modules). It then turns the bytecode for modules 
written in Python into C code (array initializers that can be turned into code objects using the marshal module) and 
creates a custom-made config file that only contains those built-in modules which are actually used in the program. 
It then compiles the generated C code and links it with the rest of the Python interpreter to form a self-contained 
binary which acts exactly like your script. 


The following packages can help with the creation of console and GUI executables: 
e Nuitka (Cross-platform) 
e Pylnstaller (Cross-platform) 
e PyOxidizer (Cross-platform) 
e cx_Freeze (Cross-platform) 
e py2app (macOS only) 
e py2exe (Windows only) 


2.1.4 Are there coding standards or a style guide for Python programs? 


Yes. The coding style required for standard library modules is documented as PEP 8. 


2.2 Core Language 


2.2.1 Why am | getting an UnboundLocalError when the variable has a value? 


It can be a surprise to get the UnboundLocalError in previously working code when it is modified by adding an 
assignment statement somewhere in the body of a function. 


This code: 


>>> x = 10 

>>> def bar(): 
print (x) 

>>> bar() 

10 


works, but this code: 


>>> x = 10 

>>> def foo(): 
print (x) 
x += 1 


results in an UnboundLocalError: 


8 Chapter 2. Programming FAQ 


Python Frequently Asked Questions, Release 3.11.1 


>>> foo() 
Traceback (most recent call last): 


UnboundLocalError: local variable 'x' referenced before assignment 


This is because when you make an assignment to a variable in a scope, that variable becomes local to that scope 
and shadows any similarly named variable in the outer scope. Since the last statement in foo assigns a new value to 
x, the compiler recognizes it as a local variable. Consequently when the earlier print (x) attempts to print the 
uninitialized local variable and an error results. 


In the example above you can access the outer scope variable by declaring it global: 


>>> x = 10 

>>> def foobar(): 
global x 
print (x) 
x t= 1 


>>> foobar () 
10 


This explicit declaration is required in order to remind you that (unlike the superficially analogous situation with class 
and instance variables) you are actually modifying the value of the variable in the outer scope: 


>>> print (x) 
11 


You can do a similar thing in a nested scope using the nonlocal keyword: 


>>> def foo(): 


x = 10 
def bar(): 
nonlocal x 
print (x) 
x += 1 
bar () 
print (x) 
>>> foo() 
10 
11 


2.2.2 What are the rules for local and global variables in Python? 


In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a value 
anywhere within the function's body, it's assumed to be a local unless explicitly declared as global. 


Though a bit surprising at first, a moment's consideration explains this. On one hand, requiring global for assigned 
variables provides a bar against unintended side-effects. On the other hand, if global was required for all global 
references, you'd be using global all the time. You'd have to declare as global every reference to a built-in function 
or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for 
identifying side-effects. 
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2.2.3 Why do lambdas defined in a loop with different values all return the same 
result? 


Assume you use a for loop to define a few different lambdas (or even plain functions), e.g.: 


>>> squares = [] 
>>> for x in range(5): 
squares.append (lambda: x**2) 


This gives you a list that contains 5 lambdas that calculate x**2. You might expect that, when called, they would 
return, respectively, 0, 1, 4, 9, and 16. However, when you actually try you will see that they all return 1 6: 


>>> squares[2] () 
16 
>>> squares [4] () 
16 


This happens because x is not local to the lambdas, but is defined in the outer scope, and it is accessed when the 
lambda is called — not when it is defined. At the end of the loop, the value of x is 4, so all the functions now return 
4**2, i.e. 16. You can also verify this by changing the value of x and see how the results of the lambdas change: 


>>> x = 8 
>>> squares[Z2] () 
64 


In order to avoid this, you need to save the values in variables local to the lambdas, so that they don’t rely on the value 
of the global x: 


>>> squares = [] 
>>> for x in range(5): 
squares.append (lambda n=x: n**2) 


Here, n=x creates a new variable n local to the lambda and computed when the lambda is defined so that it has the 
same value that x had at that point in the loop. This means that the value of n will be 0 in the first lambda, 1 in the 
second, 2 in the third, and so on. Therefore each lambda will now return the correct result: 


>>> squares[2] () 
4 

>>> squares [4] () 
16 


Note that this behaviour is not peculiar to lambdas, but applies to regular functions too. 


2.2.4 How do | share global variables across modules? 


The canonical way to share information across modules within a single program is to create a special module (often 
called config or cfg). Just import the config module in all modules of your application; the module then becomes 
available as a global name. Because there is only one instance of each module, any changes made to the module 
object get reflected everywhere. For example: 


config. py: 

x = 0 # Default value of the 'x' configuration setting 
mod.py: 

import config 

config.x = 1 

main.py: 
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import config 
import mod 
print (config.x) 


Note that using a module is also the basis for implementing the singleton design pattern, for the same reason. 


2.2.5 What are the “best practices” for using import in a module? 


In general, don't use from modulename import *. Doing so clutters the importer’s namespace, and makes it 
much harder for linters to detect undefined names. 


Import modules at the top of a file. Doing so makes it clear what other modules your code requires and avoids 
questions of whether the module name is in scope. Using one import per line makes it easy to add and delete module 
imports, but using multiple imports per line uses less screen space. 


It's good practice if you import modules in the following order: 
1. standard library modules — e.g. sys, os, argparse, re 


2. third-party library modules (anything installed in Python's site-packages directory) — e.g. dateutil, 
requests, PIL.Image 


3. locally developed modules 


It is sometimes necessary to move imports to a function or class to avoid problems with circular imports. Gordon 
McMillan says: 


Circular imports are fine where both modules use the “import <module>” form of import. They fail 
when the 2nd module wants to grab a name out of the first (“from module import name”) and the import 
is at the top level. That's because names in the 1st are not yet available, because the first module is busy 
importing the 2nd. 


In this case, if the second module is only used in one function, then the import can easily be moved into that function. 
By the time the import is called, the first module will have finished initializing, and the second module can do its 
import. 


It may also be necessary to move imports out of the top level of code if some of the modules are platform-specific. 
In that case, it may not even be possible to import all of the modules at the top of the file. In this case, importing the 
correct modules in the corresponding platform-specific code is a good option. 


Only move imports into a local scope, such as inside a function definition, if it’s necessary to solve a problem such 
as avoiding a circular import or are trying to reduce the initialization time of a module. This technique is especially 
helpful if many of the imports are unnecessary depending on how the program executes. You may also want to 
move imports into a function if the modules are only ever used in that function. Note that loading a module the first 
time may be expensive because of the one time initialization of the module, but loading a module multiple times 
is virtually free, costing only a couple of dictionary lookups. Even if the module name has gone out of scope, the 
module is probably available in sys .modules. 


2.2.6 Why are default values shared between objects? 


This type of bug commonly bites neophyte programmers. Consider this function: 


def foo(mydict={}): # Danger: shared reference to one dict for all calls 
. compute something ... 
mydict [key] = value 


return mydict 


The first time you call this function, mydict contains a single item. The second time, mydict contains two items 
because when foo () begins executing, mydict starts out with an item already in it. 
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It is often expected that a function call creates new objects for default values. This is not what happens. Default values 
are created exactly once, when the function is defined. If that object is changed, like the dictionary in this example, 
subsequent calls to the function will refer to this changed object. 


By definition, immutable objects such as numbers, strings, tuples, and None, are safe from change. Changes to 
mutable objects such as dictionaries, lists, and class instances can lead to confusion. 


Because of this feature, it is good programming practice to not use mutable objects as default values. In- 
stead, use None as the default value and inside the function, check if the parameter is None and create a new 
list/dictionary/whatever if it is. For example, don’t write: 


def foo(mydict={}): 


but: 


def foo(mydict=None): 
if mydict is None: 
mydict = {} # create a new dict for local namespace 


This feature can be useful. When you have a function that’s time-consuming to compute, a common technique is 
to cache the parameters and the resulting value of each call to the function, and return the cached value if the same 
value is requested again. This is called “memoizing”, and can be implemented like this: 


# Callers can only provide two parameters and optionally pass _cache by keyword 
def expensive (arg1, arg2, *, _cache={}): 
if (argl, arg2) in _cache: 
return _cachel (argl, arg2)] 


# Calculate the value 

result = ... expensive computation 

_cache[(argl, arg2)] = result # Store result in the cache 
return result 


You could use a global variable containing a dictionary instead of the default value; it’s a matter of taste. 


2.2.7 How can | pass optional or keyword parameters from one function to an- 
other? 


Collect the arguments using the * and ** specifiers in the function’s parameter list; this gives you the positional 
arguments as a tuple and the keyword arguments as a dictionary. You can then pass these arguments when calling 
another function by using * and **: 


def f(x, *args, **kwargs): 
kwargs['width'] = '14.3c' 


g(x, *args, **kwargs) 
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2.2.8 What is the difference between arguments and parameters? 


Parameters are defined by the names that appear in a function definition, whereas arguments are the values actually 
passed to a function when calling it. Parameters define what kind of arguments a function can accept. For example, 
given the function definition: 


def func(foo, bar=None, **kwargs): 
pass 


foo, bar and kwargs are parameters of func. However, when calling func, for example: 


func (42, bar=314, extra=somevar) 


the values 42, 314, and somevar are arguments. 


2.2.9 Why did changing list ‘y’ also change list ‘x’? 


If you wrote code like: 


>>> = [] 


= Xx 
. append (10) 


>>> 
>>> 
>>> 
[10] 
>>> x 
[10] 


RKRKK Xx 


you might be wondering why appending an element to y changed x too. 
There are two factors that produce this result: 


1) Variables are simply names that refer to objects. Doing y = x doesn’t create a copy of the list — it creates a 
new variable y that refers to the same object x refers to. This means that there is only one object (the list), and 
both x and y refer to it. 


2) Lists are mutable, which means that you can change their content. 


After the call to append (), the content of the mutable object has changed from [] to [10]. Since both the 
variables refer to the same object, using either name accesses the modified value [10]. 


If we instead assign an immutable object to x: 


>>> x = 5 # ints are immutable 

>>> y= X 

>>> x = x + 1 # 5 can't be mutated, we are creating a new object here 
>>> x 

6 

>>> y 

5 


we can see that in this case x and y are not equal anymore. This is because integers are immutable, and when we do 
x = x + 1 weare not mutating the int 5 by incrementing its value; instead, we are creating a new object (the int 
6) and assigning it to x (that is, changing which object x refers to). After this assignment we have two objects (the 
ints 6 and 5) and two variables that refer to them (x now refers to 6 but y still refers to 5). 


Some operations (for example y . append (10) and y. sort () ) mutate the object, whereas superficially similar 
operations (for example y = y + [10] and sorted (y)) create a new object. In general in Python (and in all 
cases in the standard library) a method that mutates an object will return None to help avoid getting the two types 
of operations confused. So if you mistakenly write y . sort () thinking it will give you a sorted copy of y, you'll 
instead end up with None, which will likely cause your program to generate an easily diagnosed error. 


However, there is one class of operations where the same operation sometimes has different behaviors with different 
types: the augmented assignment operators. For example, += mutates lists but not tuples or ints (a_list += 
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[1, 2, 3] is equivalent to a_list.extend([1, 2, 3]) and mutates a_list, whereas some_tuple 
+= (1, 2, 3) and some_int += 1 create new objects). 


In other words: 


e If we have a mutable object (list, dict, set, etc.), we can use some specific operations to mutate it and 
all the variables that refer to it will see the change. 


e If we have an immutable object (str, int, tuple, etc.), all the variables that refer to it will always see the 
same value, but operations that transform that value into a new value always return a new object. 


If you want to know if two variables refer to the same object or not, you can use the is operator, or the built-in 
function id (). 


2.2.10 How do | write a function with output parameters (call by reference)? 


Remember that arguments are passed by assignment in Python. Since assignment just creates references to objects, 
there's no alias between an argument name in the caller and callee, and so no call-by-reference per se. You can achieve 
the desired effect in a number of ways. 


1) By returning a tuple of the results: 


>>> def funci(a, b): 


a = 'new-value' # a and b are local names 
b=b+1 # assigned to new objects 
return a, b # return new values 

>>> x, y = 'old-value', 99 


>>> funcl(x, y) 
('new-value', 100) 


This is almost always the clearest solution. 
2) By using global variables. This isn't thread-safe, and is not recommended. 


3) By passing a mutable (changeable in-place) object: 


>>> def func2 (a): 


a[0] = 'new-value' # 'a' references a mutable list 
a[1] = a[1] + 1 # changes a shared object 
>>> args = ['old-value', 99] 


>>> func2 (args) 
>>> args 
['new-value', 100] 


4) By passing in a dictionary that gets mutated: 


>>> def func3(args): 


args['a'] = 'new-value' # args is a mutable dictionary 
args['b'] = args['b'] + 1 # change it in-place 
>>> args = {'a': 'old-value', 'b': 99} 


>>> func3 (args) 
>>> args 
{'a': 'new-value', 'b': 100} 


5) Or bundle up values in a class instance: 


>>> class Namespace: 
def __init_ (self, /, **args): 
for key, value in args.items(): 
setattr (self, key, value) 


(continues on next page) 
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(continued from previous page) 


>>> def func! (args): 
args.a = 'new-value' # args is a mutable Namespace 
args.b = args.b + 1 # change object in-place 


>>> args = Namespace (a='old-value', b=99) 
>>> func4 (args) 

>>> vars (args) 

{'a': 'new-value', 'b': 100} 


There’s almost never a good reason to get this complicated. 


Your best choice is to return a tuple containing the multiple results. 


2.2.11 How do you make a higher order function in Python? 


You have two choices: you can use nested scopes or you can use callable objects. For example, suppose you wanted 
to define linear (a,b) which returns a function f (x) that computes the value a*x+b. Using nested scopes: 


def linear(a, b): 
def result (x): 
return a * x + b 
return result 


Or using a callable object: 


class linear: 


def _ init_ (self, a, b): 
self.a, self.b = a, b 


def __call_ (self, x): 
return self.a * x + self.b 


In both cases, 


taxes = linear(0.3, 2) 


gives a callable object where taxes (10e6) == 0.3 * 10e6 + 2. 


The callable object approach has the disadvantage that it is a bit slower and results in slightly longer code. However, 
note that a collection of callables can share their signature via inheritance: 


class exponential (linear): 
# _ init__ inherited 
def __ call_ (self, x): 
return self.a * (x ** self.b) 


Object can encapsulate state for several methods: 


class counter: 


value = 0 


def set (self, x): 
self.value = x 


def up(self): 
self.value = self.value + 1 


(continues on next page) 
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(continued from previous page) 


def down (self): 


self.value = self.value - 1 
count = counter () 
inc, dec, reset = count.up, count.down, count.set 


Here inc (), dec () and reset () act like functions which share the same counting variable. 


2.2.12 How do | copy an object in Python? 


In general, try copy. copy () or copy . deepcopy () for the general case. Not all objects can be copied, but 
most can. 


Some objects can be copied more easily. Dictionaries have a copy () method: 


newdict = olddict.copy() 


Sequences can be copied by slicing: 


new_1 = 1[:] 


2.2.13 How can I find the methods or attributes of an object? 


For an instance x of a user-defined class, dir (x) returns an alphabetized list of the names containing the instance 
attributes and methods and attributes defined by its class. 


2.2.14 How can my code discover the name of an object? 


Generally speaking, it can’t, because objects don't really have names. Essentially, assignment always binds a name to 
a value; the same is true of def and class statements, but in that case the value is a callable. Consider the following 
code: 


>>> class A: 
pass 
>>> B=A 
>>> a = B() 
>>> b=a 
>>> print (b) 
<__main__.A object at 0x16D07CC> 
>>> print (a) 
<__main__.A object at 0x16D07CC> 


Arguably the class has a name: even though it is bound to two names and invoked through the name B the created 
instance is still reported as an instance of class A. However, it is impossible to say whether the instance’s name is a 
or b, since both names are bound to the same value. 


Generally speaking it should not be necessary for your code to “know the names” of particular values. Unless you are 
deliberately writing introspective programs, this is usually an indication that a change of approach might be beneficial. 


In comp.lang.python, Fredrik Lundh once gave an excellent analogy in answer to this question: 


The same way as you get the name of that cat you found on your porch: the cat (object) itself cannot 
tell you its name, and it doesn’t really care — so the only way to find out what it’s called is to ask all your 
neighbours (namespaces) if it’s their cat (object)... 


....and don’t be surprised if you'll find that it’s known by many names, or no name at all! 
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2.2.15 What's up with the comma operator's precedence? 


Comma is not an operator in Python. Consider this session: 


>>> wan in "h" j "an 
(False, 'a') 


Since the comma is not an operator, but a separator between expressions the above is evaluated as if you had entered: 


cua in NH"), wou 


"a" in (Apr, nat 


The same is true of the various assignment operators (=, += etc). They are not truly operators but syntactic delimiters 
in assignment statements. 


2.2.16 Is there an equivalent of C's “?:” ternary operator? 


Yes, there is. The syntax is as follows: 


[on_true] if [expression] else [on_false] 


X, y = 50, 25 
small = x if x < y else y 


Before this syntax was introduced in Python 2.5, a common idiom was to use logical operators: 


[expression] and [on_true] or [on_false] 


However, this idiom is unsafe, as it can give wrong results when on_true has a false boolean value. Therefore, it is 
always better to use the... if ... else ... form. 


2.2.17 Is it possible to write obfuscated one-liners in Python? 


Yes. Usually this is done by nesting 1ambda within lambda. See the following three examples, slightly adapted 
from Ulf Bartelt: 


from functools import reduce 


# Primes < 1000 
print (list (filter (None, map (lambda y:y*reduce (lambda x,y:x*y!=0, 
map (lambda x,y=y:y%x, range (2,int (pow(y,0.5)+1))),1), range (2,1000))))) 


# First 10 Fibonacci numbers 
print (list (map (lambda x, f=lambda x, f: (f (x-1,f)+f(x-2,f)) if x>1 else 1: 
f(x,f), range(10)))) 


# Mandelbrot set 

print ((lambda Ru, Ro, lu, lo, IM,Sx,Sy:reduce (lambda x,y:x+'\n'+y,map (lambda y, 
Tu=Iu, Io=Io, Ru=Ru, Ro=Ro, Sy=Sy, L=lambda yc, lu=lu, Io=Io, Ru=Ru, Ro=Ro,i=IM, 
Sx=Sx, Sy=Sy:reduce (lambda x,y:x+y,map (lambda x, xc=Ru, yc=yc, Ru=Ru, Ro=Ro, 
i=i,Sx=Sx,F=lambda xc,yc,x,y,k,f=lambda xc,yc,x,y,k,f: (k<=0)or (x*x+y*y 
>=4.0) or 1+f(xc,yc, x*x-y*ytxc,2.0*x*ytyc,k-1,f):f(xc,yc,x,y,k,f£) :chr ( 


64+F (Ru+x* (Ro-Ru) /Sx, yc, 0,0,i)),range(Sx))):L(Iuty* (Io-Iu) /Sy), range (Sy 
(Hae O27; Lal Liz, 30, 80, 24)) 

# \ AN J | / /__ lines on screen 

# V V / / columns on screen 


(continues on next page) 
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(continued from previous page) 


# / / / maximum of "iterations" 


$ 
== 


range on y axis 
# / range on x axis 


Don’t try this at home, kids! 


2.2.18 What does the slash(/) in the parameter list of a function mean? 


A slash in the argument list of a function denotes that the parameters prior to it are positional-only. Positional-only 
parameters are the ones without an externally usable name. Upon calling a function that accepts positional-only 
parameters, arguments are mapped to parameters based solely on their position. For example, divmod () is a 
function that accepts positional-only parameters. Its documentation looks like this: 


>>> help (divmod) 
Help on built-in function divmod in module builtins: 


divmod(x, y, /) 
Return the tuple (x//y, x%y). Invariant: div*y + mod == x. 


The slash at the end of the parameter list means that both parameters are positional-only. Thus, calling divmod () 
with keyword arguments would lead to an error: 


>>> divmod(x=3, y=4) 

Traceback (most recent call last): 

File "<stdin>", line 1, in <module> 
ypeError: divmod() takes no keyword arguments 


2.3 Numbers and strings 


2.3.1 How do | specify hexadecimal and octal integers? 


To specify an octal digit, precede the octal value with a zero, and then a lower or uppercase “o”. For example, to set 
the variable “a” to the octal value “10” (8 in decimal), type: 


>>> a = 0010 
>>> a 
8 


CL 


Hexadecimal is just as easy. Simply precede the hexadecimal number with a zero, and then a lower or uppercase “x”. 
Hexadecimal digits can be specified in lower or uppercase. For example, in the Python interpreter: 


>>> a = 0xa5 
>>> a 
165 
>>> b 
>>> b 
178 


OXB2 
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2.3.2 Why does -22 // 10 return -3? 


It’s primarily driven by the desire that i % 3 have the same sign as 3. If you want that, and also want: 


i== (1 // j) * J + (1 5% J) 


then integer division has to return the floor. C also requires that identity to hold, and then compilers that truncate i 
// j need to make i % j have the same sign as i. 


There are few real use cases for i % j when j is negative. When j is positive, there are many, and in virtually all 
of them it’s more useful for i % j tobe >= 0. If the clock says 10 now, what did it say 200 hours ago? -190 $ 
12 == 2isuseful;-190 $ 12 == -10 isa bug waiting to bite. 


2.3.3 How do | get int literal attribute instead of SyntaxError? 


Trying to lookup an int literal attribute in the normal manner gives a SyntaxError because the period is seen 
as a decimal point: 


>>> 1, class. 
File "<stdin>", line 1 
1 elass 


A 


SyntaxError: invalid decimal literal 


The solution is to separate the literal from the period with either a space or parentheses. 


>>> 1 ._ class 
<class 'int'> 
55> (1). class __ 


<class 'int'> 


2.3.4 How do | convert a string to a number? 


For integers, use the built-in int () type constructor, e.g. int ('144') == 144. Similarly, float () converts 
to floating-point, e.g. float ('144') == 144.0. 


By default, these interpret the number as decimal, so that int ('0144') == 144 holds true, and 
int ('0x144"') raises ValueError. int (string, base) takes the base to convert from as a second op- 
tional argument, so int ( '0x144', 16) == 324. If the base is specified as 0, the number is interpreted using 
Python’s rules: a leading ‘Oo’ indicates octal, and ‘Ox’ indicates a hex number. 


Do not use the built-in function eval () if all you need is to convert strings to numbers. eval () will be significantly 
slower and it presents a security risk: someone could pass you a Python expression that might have unwanted side 
effects. For example, someone could pass __import__('os') .system("rm -rf $HOME") which would 
erase your home directory. 


eval () also has the effect of interpreting numbers as Python expressions, so that e.g. eval ('09') gives a syntax 
error because Python does not allow leading ‘0’ in a decimal number (except ‘0’). 
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2.3.5 How do | convert a number to a string? 


To convert, e.g., the number 144 to the string '144', use the built-in type constructor str () . If you want a hex- 
adecimal or octal representation, use the built-in functions hex () or oct (). For fancy formatting, see the f-strings 
and formatstrings sections, e.g. "4:04d)". format (144) yields '0144' and "4: .3£)".format (1.0/ 
3.0) yields '0.333'. 


2.3.6 How do | modify a string in place? 


You can’t, because strings are immutable. In most situations, you should simply construct a new string from the various 
parts you want to assemble it from. However, if you need an object with the ability to modify in-place unicode data, 
try using an io. St ringIO object or the array module: 


>>> import io 

>>> s = "Hello, world" 
>>> sio = io.StringI0O(s) 
>>> sio.getvalue() 
"Hello, world' 

>>> sio.seek(7) 

7 

>>> sio.write("there!") 
6 

>>> sio.getvalue() 
"Hello, there!' 


>>> import array 

>>> a = array.array('u', s) 
>>> print (a) 

array('u', 'Hello, world') 
>>> a[0] = 'y" 

>>> print (a) 

array('u', 'yello, world') 
>>> a.tounicode () 

'yello, world' 


2.3.7 How do | use strings to call functions/methods? 


There are various techniques. 


e The best is to use a dictionary that maps strings to functions. The primary advantage of this technique is that 
the strings do not need to match the names of the functions. This is also the primary technique used to emulate 
a case construct: 


def a(): 
pass 


def b(): 
pass 


dispatch = {'go': a, 'stop': b} # Note lack of parens for funcs 


dispatch[get_input () ] () # Note trailing parens to call function 


e Use the built-in function getattr (): 


import foo 
getattr (foo, 'bar') () 


Note that getattr () works on any object, including classes, class instances, modules, and so on. 
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This is used in several places in the standard library, like this: 


class Foo: 
def do_foo(self): 


def do_bar (self): 


f = getattr(foo_instance, 'do_' + opname) 
£() 


e Use locals () to resolve the function name: 


def myFunc(): 
print ("hello") 


fname = "myFunc" 
f = locals() [fname] 
£() 


2.3.8 Is there an equivalent to Perl’s chomp() for removing trailing newlines from 
strings? 


You can use S.rstrip("\r\n") to remove all occurrences of any line terminator from the end of the string S 
without removing other trailing whitespace. If the string S represents more than one line, with several empty lines at 
the end, the line terminators for all the blank lines will be removed: 


>>> lines = ("line 1 \r\n" 
" Win" 

día "\r\n") 

>>> lines.rstrip("\n\r") 

‘line 1 ' 


Since this is typically only desired when reading text one line at a time, using S.rstrip () this way works well. 


2.3.9 Is there a scanf() or sscanf() equivalent? 


Not as such. 


For simple input parsing, the easiest approach is usually to split the line into whitespace-delimited words using the 
split () method of string objects and then convert decimal strings to numeric values using int () or float (). 
split () supports an optional “sep” parameter which is useful if the line uses something other than whitespace as 
a separator. 


For more complicated input parsing, regular expressions are more powerful than C's sscanf and better suited for 
the task. 
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2.3.10 What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean? 


See the unicode-howto. 


2.3.11 Can I end a raw string with an odd number of backslashes? 


A raw string ending with an odd number of backslashes will escape the string's quote: 


>>> r'C:\this\will\not\work\! 
File "<stdin>", line 1 
r'C:\this\will\not\work\'! 


A 


SyntaxError: unterminated string literal (detected at line 1) 


There are several workarounds for this. One is to use regular strings and double the backslashes: 


>>> 'C:\\this\\will\\work\\'! 
"C:\\this\\will\\work\\! 


Another is to concatenate a regular string containing an escaped backslash to the raw string: 


55> 2 'C?\this\will\werk" "NX" 
'C:\\this\\will\\work\\! 


It is also possible to use os. path. join () to append a backslash on Windows: 


>>> os.path.join(r'C:\this\will\work', '') 
"C:\\this\\will\\work\\! 


Note that while a backslash will “escape” a quote for the purposes of determining where the raw string ends, no 
escaping occurs when interpreting the value of the raw string. That is, the backslash remains present in the value of 
the raw string: 


>>> r'backslash\'preserved' 
"backslash\\'preserved" 


Also see the specification in the language reference. 


2.4 Performance 


2.4.1 My program is too slow. How do | speed it up? 


That’s a tough one, in general. First, here are a list of things to remember before diving further: 


Performance characteristics vary across Python implementations. This FAQ focuses on CPython. 


Behaviour can vary across operating systems, especially when talking about I/O or multi-threading. 


You should always find the hot spots in your program before attempting to optimize any code (see the profile 
module). 


Writing benchmark scripts will allow you to iterate quickly when searching for improvements (see the t imeit 
module). 


It is highly recommended to have good code coverage (through unit testing or any other technique) before 
potentially introducing regressions hidden in sophisticated optimizations. 


That being said, there are many tricks to speed up Python code. Here are some general principles which go a long 
way towards reaching acceptable performance levels: 
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Making your algorithms faster (or changing to faster ones) can yield much larger benefits than trying to sprinkle 
micro-optimization tricks all over your code. 


Use the right data structures. Study documentation for the bltin-types and the collections module. 


When the standard library provides a primitive for doing something, it is likely (although not guaranteed) to 
be faster than any alternative you may come up with. This is doubly true for primitives written in C, such as 
builtins and some extension types. For example, be sure to use either the list .sort () built-in method or 
the related sorted () function to do sorting (and see the sortinghowto for examples of moderately advanced 
usage). 


Abstractions tend to create indirections and force the interpreter to work more. If the levels of indirection 
outweigh the amount of useful work done, your program will be slower. You should avoid excessive abstraction, 
especially under the form of tiny functions or methods (which are also often detrimental to readability). 


If you have reached the limit of what pure Python can allow, there are tools to take you further away. For example, 
Cython can compile a slightly modified version of Python code into a C extension, and can be used on many different 
platforms. Cython can take advantage of compilation (and optional type annotations) to make your code significantly 
faster than when interpreted. If you are confident in your C programming skills, you can also write a C extension 
module yourself. 


See also: 


The wiki page devoted to performance tips. 


2.4.2 What is the most efficient way to concatenate many strings together? 
str and bytes objects are immutable, therefore concatenating many strings together is inefficient as each concate- 
nation creates a new object. In the general case, the total runtime cost is quadratic in the total string length. 


To accumulate many st r objects, the recommended idiom is to place them into a list and call str.join () at the 
end: 


chunks = [] 

for s in my_strings: 
chunks.append(s) 

result = ''.join(chunks) 


(another reasonably efficient idiom is to use io. St ringI0) 


To accumulate many bytes objects, the recommended idiom is to extend a bytearray object using in-place 
concatenation (the += operator): 


result = bytearray() 
for b in my_bytes_objects: 
result += b 


2.5 Sequences (Tuples/Lists) 


2.5.1 How do | convert between tuples and lists? 


The type constructor tuple (seq) converts any sequence (actually, any iterable) into a tuple with the same items 
in the same order. 


For example, tuple ([1, 2, 3]) yields (1, 2, 3) andtuple('abc') yields ('a', 'b', 'c'). If 
the argument is a tuple, it does not make a copy but returns the same object, so it is cheap to call tuple () when 
you aren’t sure that an object is already a tuple. 


The type constructor 1ist (seq) converts any sequence or iterable into a list with the same items in the same order. 
For example, list ( (1, 2, 3)) yields [1, 2, 3] andlist('abc') yields ['a', 'b', 'c']. Ifthe 
argument is a list, it makes a copy just like seq [ : ] would. 
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2.5.2 What's a negative index? 


Python sequences are indexed with positive numbers and negative numbers. For positive numbers 0 is the first index 
1 is the second index and so forth. For negative indices -1 is the last index and -2 is the penultimate (next to last) 
index and so forth. Think of seq[-n] as the same as seq[ len (seq) -n]. 


Using negative indices can be very convenient. For example S [ :-1] is all of the string except for its last character, 
which is useful for removing the trailing newline from a string. 


2.5.3 How do | iterate over a sequence in reverse order? 


Use the reversed () built-in function: 


for x in reversed (sequence): 
# do something with x ... 


This won’t touch your original sequence, but build a new copy with reversed order to iterate over. 


2.5.4 How do you remove duplicates from a list? 


See the Python Cookbook for a long discussion of many ways to do this: 
https://code.activestate.com/recipes/52560/ 


If you don’t mind reordering the list, sort it and then scan from the end of the list, deleting duplicates as you go: 


if mylist: 
mylist.sort () 
last = mylist[-1] 
for i in range (len (mylist)-2, -1, -1): 
if last == mylist[i]: 
del mylist[i] 
else: 
last = mylist [i] 


If all elements of the list may be used as set keys (i.e. they are all hashable) this is often faster 


mylist = list (set (mylist) ) 


This converts the list into a set, thereby removing duplicates, and then back into a list. 


2.5.5 How do you remove multiple items from a list 


As with removing duplicates, explicitly iterating in reverse with a delete condition is one possibility. However, it is 
easier and faster to use slice replacement with an implicit or explicit forward iteration. Here are three variations.: 


mylist[:] = filter(keep_function, mylist) 
mylist[:] = (x for x in mylist if keep_condition) 
mylist[:] = [x for x in mylist if keep_condition] 


The list comprehension may be fastest. 
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2.5.6 How do you make an array in Python? 


Use a list: 


["this", f wis "ant. "array"] 


Lists are equivalent to C or Pascal arrays in their time complexity; the primary difference is that a Python list can 
contain objects of many different types. 


The array module also provides methods for creating arrays of fixed types with compact representations, but they 
are slower to index than lists. Also note that NumPy and other third party packages define array-like structures with 
various characteristics as well. 


To get Lisp-style linked lists, you can emulate cons cells using tuples: 


lisp_list = ("like", Cheha", ("example", None) ) ) 


If mutability is desired, you could use lists instead of tuples. Here the analogue of a Lisp car is lisp_list [0] 
and the analogue of cdr is 1isp_list [1]. Only do this if you're sure you really need to, because it’s usually a lot 
slower than using Python lists. 


2.5.7 How do | create a multidimensional list? 


You probably tried to make a multidimensional array like this: 


>>> A = [[None] * 2] * 3 


This looks correct if you print it: 


>>> A 
[[None, None], [None, None], [None, None] ] 


But when you assign a value, it shows up in multiple places: 


>>> A[O0] [0] = 5 
>>> A 
[[5, None], [5, None], [5, None] ] 


The reason is that replicating a list with * doesn’t create copies, it only creates references to the existing objects. The 
*3 creates a list containing 3 references to the same list of length two. Changes to one row will show in all rows, 
which is almost certainly not what you want. 


The suggested approach is to create a list of the desired length first and then fill in each element with a newly created 
list: 


A = [None] * 3 
for i in range(3): 
Afi] = [None] * 2 


This generates a list containing 3 different lists of length two. You can also use a list comprehension: 


w, h = 2, 3 
A = [[None] * w for i in range(h) ] 


Or, you can use an extension that provides a matrix datatype; NumPy is the best known. 
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2.5.8 How do | apply a method or function to a sequence of objects? 


To call a method or function and accumulate the return values is a list, a list comprehension is an elegant solution: 


result = [obj.method() for obj in mylist] 


result = [function(obj) for obj in mylist] 


To just run the method or function without saving the return values, a plain for loop will suffice: 


for obj in mylist: 
obj.method() 


for obj in mylist: 
function (obj) 


2.5.9 Why does a_tuple[i] += [‘item’] raise an exception when the addition 
works? 


This is because of a combination of the fact that augmented assignment operators are assignment operators, and the 
difference between mutable and immutable objects in Python. 


This discussion applies in general when augmented assignment operators are applied to elements of a tuple that point 
to mutable objects, but we'll use a 1ist and += as our exemplar. 


If you wrote: 


>>> a_tuple = (1, 2) 
>>> a_tuple[0] += 1 
Traceback (most recent call last): 


ypeError: 'tuple' object does not support item assignment 


The reason for the exception should be immediately clear: 1 is added to the object a_tuple[0] points to (1), 
producing the result object, 2, but when we attempt to assign the result of the computation, 2, to element 0 of the 
tuple, we get an error because we can’t change what an element of a tuple points to. 


Under the covers, what this augmented assignment statement is doing is approximately this: 


>>> result = a_tuple[0] + 1 
>>> a_tuple[0] = result 
Traceback (most recent call last): 


ypeError: 'tuple' object does not support item assignment 


It is the assignment part of the operation that produces the error, since a tuple is immutable. 


When you write something like: 


>>> a_tuple = (['foo'], 'bar') 
>>> a_tuple[0] += ['item'] 
Traceback (most recent call last): 


TypeError: 'tuple' object does not support item assignment 


The exception is a bit more surprising, and even more surprising is the fact that even though there was an error, the 
append worked: 


>>> a_tuple[0] 
['foo't, 'item'] 
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To see why this happens, you need to know that (a) if an object implements an __iadd__ () magic method, it 
gets called when the += augmented assignment is executed, and its return value is what gets used in the assignment 
statement; and (b) for lists,___ iadd__ () is equivalent to calling extend () on the list and returning the list. That's 
why we say that for lists, += is a “shorthand” for list .extend (): 


>>> a_list = [] 
>>> a list += [1] 
>>> a_ list 


[1] 


This is equivalent to: 


>>> result = a_list.__iadd__([1]) 
>>> a_list = result 


The object pointed to by a_list has been mutated, and the pointer to the mutated object is assigned back to a_list. 
The end result of the assignment is a no-op, since it is a pointer to the same object that a_1ist was previously 
pointing to, but the assignment still happens. 


Thus, in our tuple example what is happening is equivalent to: 


>>> result = a_tuple[0].__iadd__(['item']) 
>>> a_tuple[0] = result 
Traceback (most recent call last): 


ypeError: 'tuple' object does not support item assignment 


The __iadd__ () succeeds, and thus the list is extended, but even though result points to the same object that 
a_tuple[0] already points to, that final assignment still results in an error, because tuples are immutable. 


2.5.10 I want to do a complicated sort: can you do a Schwartzian Transform in 
Python? 


The technique, attributed to Randal Schwartz of the Perl community, sorts the elements of a list by a metric which 
maps each element to its “sort value”. In Python, use the key argument for the list .sort () method: 


Isorted = L[:] 
Isorted.sort (key=lambda s: int(s[10:15])) 


2.5.11 How can | sort one list by values from another list? 


Merge them into an iterator of tuples, sort the resulting list, and then pick out the element you want. 


>>> listi = ["what", "I'm", "sorting", "by"] 

>>> list2 = ["something", "else", "to", "sort"] 

>>> pairs = zip(listi, list2) 

>>> pairs = sorted(pairs) 

>>> pairs 

[("I'm", 'else'), ('by', 'sort'), ('sorting', 'to'), ('what', 'something')] 
>>> result = [x[1] for x in pairs] 


>>> result 
['else', 'sort', 'to', 'something'] 
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2.6 Objects 


2.6.1 What is a class? 


A class is the particular object type created by executing a class statement. Class objects are used as templates to 
create instance objects, which embody both the data (attributes) and code (methods) specific to a datatype. 


A class can be based on one or more other classes, called its base class(es). It then inherits the attributes and meth- 
ods of its base classes. This allows an object model to be successively refined by inheritance. You might have a 
generic Mailbox class that provides basic accessor methods for a mailbox, and subclasses such as MboxMailbox, 
MaildirMailbox, OutlookMailbox that handle various specific mailbox formats. 


2.6.2 What is a method? 


A method is a function on some object x that you normally call as x.name (arguments...). Methods are 
defined as functions inside the class definition: 


class C: 
def meth(self, arg): 
return arg * 2 + self.attribute 


2.6.3 What is self? 


Self is merely a conventional name for the first argument of a method. A method defined as meth (self, a, b, 
c) should be called as x.meth(a, b, c) for some instance x of the class in which the definition occurs; the 
called method will think it is called as meth (x, a, b, c). 


See also Why must ‘self’ be used explicitly in method definitions and calls?. 


2.6.4 How do | check if an object is an instance of a given class or of a subclass 
of it? 


Use the built-in function isinstance (obj, cls). You can check if an object is an instance of any of a number 
of classes by providing a tuple instead of a single class, e.g. isinstance (obj, (classi, class2, 

) ), and can also check whether an object is one of Python's built-in types, e.g. isinstance (obj, str) or 
isinstance(obj, (int, float, complex)). 


Note that isinstance () also checks for virtual inheritance from an abstract base class. So, the test will return 
True for a registered class even if hasn't directly or indirectly inherited from it. To test for “true inheritance”, scan 
the MRO of the class: 


from collections.abc import Mapping 


class P: 
pass 


class C(P): 
pass 


Mapping.register (P) 


>>> c = C() 

>>> isinstance(c, C) # direct 
True 

>>> isinstance(c, P) # indirect 
True 


(continues on next page) 
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(continued from previous page) 


>>> isinstance(c, Mapping) # virtual 
True 


# Actual inheritance chain 
>>> type(c).__mro__ 
(<class 'C'>, <class 'P'>, <class 'object'>) 


# Test for "true inheritance" 
>>> Mapping in type(c).__mro__ 
False 


Note that most programs do not use isinstance () on user-defined classes very often. If you are developing the 
classes yourself, a more proper object-oriented style is to define methods on the classes that encapsulate a particular 
behaviour, instead of checking the object's class and doing a different thing based on what class it is. For example, if 
you have a function that does something: 


def search (obj): 
if isinstance (obj, Mailbox): 
# code to search a mailbox 
elif isinstance (obj, Document): 
: # code to search a document 
elif 


A better approach is to define a search () method on all the classes and just call it: 


class Mailbox: 
def search(self): 
# code to search a mailbox 


class Document: 
def search(self): 


# code to search a document 


obj .search () 


2.6.5 What is delegation? 


Delegation is an object oriented technique (also called a design pattern). Let’s say you have an object x and want to 
change the behaviour of just one of its methods. You can create a new class that provides a new implementation of 
the method you're interested in changing and delegates all other methods to the corresponding method of x. 


Python programmers can easily implement delegation. For example, the following class implements a class that 
behaves like a file but converts all written data to uppercase: 


class UpperOut: 


def init__(self, outfile): 
self._outfile = outfile 


def write(self, s): 
self._outfile.write(s.upper () ) 


def __getattr__(self, name): 
return getattr(self._outfile, name) 


Here the UpperOut class redefines the write () method to convert the argument string to uppercase before 
calling the underlying self._outfile.write() method. All other methods are delegated to the underlying 
self ._outfile object. The delegation is accomplished via the ___ get at tr___() method; consult the language 
reference for more information about controlling attribute access. 
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Note that for more general cases delegation can get trickier. When attributes must be set as well as retrieved, 
the class must define a __setattr__ () method too, and it must do so carefully. The basic implementation 
of __setattr__() is roughly equivalent to the following: 


class X: 


def _ setattr_ (self, name, value): 
self. _ _dict_ [name] = value 


Most__setattr__ () implementations must modify self.__dict__ to store local state for self without caus- 
ing an infinite recursion. 


2.6.6 How do | call a method defined in a base class from a derived class that 
extends it? 


Use the built-in super () function: 


class Derived (Base): 
def meth (self): 
super () .meth () # calls Base.meth 


In the example, super () will automatically determine the instance from which it was called (the se 1 f value), look 
up the method resolution order (MRO) with type (self) .__mro__, and return the next in line after Derived 
in the MRO: Base. 


2.6.7 How can | organize my code to make it easier to change the base class? 


You could assign the base class to an alias and derive from the alias. Then all you have to change is the value assigned 
to the alias. Incidentally, this trick is also handy if you want to decide dynamically (e.g. depending on availability of 
resources) which base class to use. Example: 


class Base: 


BaseAlias = Base 


class Derived (BaseAlias): 


2.6.8 How do | create static class data and static class methods? 


Both static data and static methods (in the sense of C++ or Java) are supported in Python. 


For static data, simply define a class attribute. To assign a new value to the attribute, you have to explicitly use the 
class name in the assignment: 


class C: 
count = 0 # number of times C.__init__ called 


def __ init_ (self): 
C.count = C.count + 1 


def getcount (self): 
return C.count # or return self.count 


c.count also refers to C. count for any c such that isinstance (c, C) holds, unless overridden by c itself 
or by some class on the base-class search path from c.__class__ back to C. 
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Caution: within a method of C, an assignment like self .count = 42 creates a new and unrelated instance named 
“count” in sel £”s own dict. Rebinding of a class-static data name must always specify the class whether inside a 
method or not: 


C.count = 314 


Static methods are possible: 


class C: 
@staticmethod 
def static(argl, arg2, arg3): 
# No 'self' parameter! 


However, a far more straightforward way to get the effect of a static method is via a simple module-level function: 


def getcount(): 
return C.count 


If your code is structured so as to define one class (or tightly related class hierarchy) per module, this supplies the 
desired encapsulation. 


2.6.9 How can | overload constructors (or methods) in Python? 


This answer actually applies to all methods, but the question usually comes up first in the context of constructors. 


In C++ you’d write 


class C { 
C() { cout << "No arguments\n"; } 
C(int i) { cout << "Argument is "<< i << "An"; } 


In Python you have to write a single constructor that catches all cases using default arguments. For example: 


class C: 
def __init_ (self, i=None): 
if i is None: 
print ("No arguments") 
else: 
print ("Argument is", i) 


This is not entirely equivalent, but close enough in practice. 


You could also try a variable-length argument list, e.g. 


def __ init__ (self, *args): 


The same approach works for all method definitions. 
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2.6.10 I try to use __spam and I get an error about _SomeClassName__spam. 


Variable names with double leading underscores are “mangled” to provide a simple but effective way to define class 
private variables. Any identifier of the form __ spam (at least two leading underscores, at most one trailing under- 
score) is textually replaced with _classname__ spam, where classname is the current class name with any 
leading underscores stripped. 


This doesn’t guarantee privacy: an outside user can still deliberately access the “_classname__ spam” attribute, and 
private values are visible in the object's__dict__. Many Python programmers never bother to use private variable 
names at all. 


2.6.11 My class defines __del__ but it is not called when I delete the object. 


There are several possible reasons for this. 


The del statement does not necessarily call ___de1___() — it simply decrements the object's reference count, and 
if this reaches zero__del__() is called. 


If your data structures contain circular links (e.g. a tree where each child has a parent reference and each parent has 
a list of children) the reference counts will never go back to zero. Once in a while Python runs an algorithm to detect 
such cycles, but the garbage collector might run some time after the last reference to your data structure vanishes, so 
your__del__ () method may be called at an inconvenient and random time. This is inconvenient if you're trying 
to reproduce a problem. Worse, the order in which object's__ de 1___ () methods are executed is arbitrary. You can 
run gc.collect () to force a collection, but there are pathological cases where objects will never be collected. 


Despite the cycle collector, it’s still a good idea to define an explicit close () method on objects to be called 
whenever you’re done with them. The close () method can then remove attributes that refer to subobjects. Don’t 
call__del___() directly -__de1___() should call close() and close () should make sure that it can be 
called more than once for the same object. 


Another way to avoid cyclical references is to use the weakref module, which allows you to point to objects without 
incrementing their reference count. Tree data structures, for instance, should use weak references for their parent and 
sibling references (if they need them!). 


Finally, if your__ de 1___() method raises an exception, a warning message is printed to sys. stderr. 


2.6.12 How do | get a list of all instances of a given class? 


Python does not keep track of all instances of a class (or of a built-in type). You can program the class's constructor 
to keep track of all instances by keeping a list of weak references to each instance. 


2.6.13 Why does the result of id () appear to be not unique? 


The id () builtin returns an integer that is guaranteed to be unique during the lifetime of the object. Since in 
CPython, this is the object’s memory address, it happens frequently that after an object is deleted from memory, the 
next freshly created object is allocated at the same position in memory. This is illustrated by this example: 


>>> id(1000) 
13901272 
>>> id(2000) 
13901272 


The two ids belong to different integer objects that are created before, and deleted immediately after execution of the 
id () call. To be sure that objects whose id you want to examine are still alive, create another reference to the object: 


>>> a = 1000; b = 2000 
>>> id(a) 
13901272 
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>>> id(b) 
13891296 


2.6.14 When can I rely on identity tests with the is operator? 


The is operator tests for object identity. The testa is b is equivalent to id (a) == id(b). 


The most important property of an identity test is that an object is always identical to itself, a is a always returns 
True. Identity tests are usually faster than equality tests. And unlike equality tests, identity tests are guaranteed to 
return a boolean True or False. 


However, identity tests can only be substituted for equality tests when object identity is assured. Generally, there are 
three circumstances where identity is guaranteed: 


1) Assignments create new names but do not change object identity. After the assignment new = ola, it is guar- 
anteed that new is old. 


2) Putting an object in a container that stores object references does not change object identity. After the list assign- 
ment s[0] = x, itis guaranteed that s[0] is x. 


3) If an object is a singleton, it means that only one instance of that object can exist. After the assignments a = 
None and lb = None, it is guaranteed that a is b because None isa singleton. 


In most other circumstances, identity tests are inadvisable and equality tests are preferred. In particular, identity tests 
should not be used to check constants such as int and str which aren't guaranteed to be singletons: 


>>> a = 1000 
>>> b = 500 

>>> c = b + 500 
>>> a isc 
False 

>>> a = 'Python' 
>>> b = 'Py' 


>>> a isc 
False 


Likewise, new instances of mutable containers are never identical: 


>>> a = [] 
>>> b = [] 
>>> a is b 
False 


In the standard library code, you will see several common patterns for correctly using identity tests: 


1) As recommended by PEP 8, an identity test is the preferred way to check for None. This reads like plain English 
in code and avoids confusion with other objects that may have boolean values that evaluate to false. 


2) Detecting optional arguments can be tricky when None is a valid input value. In those situations, you can create 
a singleton sentinel object guaranteed to be distinct from other objects. For example, here is how to implement a 
method that behaves like dict .pop (): 


_sentinel = object () 


def pop(self, key, default=_sentinel): 
if key in self: 
value = self [key] 
del self [key] 
return value 
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if default is _sentinel: 
raise KeyError (key) 
return default 


3) Container implementations sometimes need to augment equality tests with identity tests. This prevents the code 


from being confused by objects such as float ('NaN' ) that are not equal to themselves. 


For example, here is the implementation of collections.abc.Sequence.__contains__(): 


def _ contains_ (self, value): 
for v in self: 
if v is value or v == valu 
return True 
return False 


2.6.15 How can a subclass control what data is stored in an immutable in- 


stance? 


When subclassing an immutable type, override the __new___ () method instead of the__init__ () method. The 
latter only runs after an instance is created, which is too late to alter data in an immutable instance. 


All of these immutable classes have a different signature than their parent class: 


from datetime import date 


class FirstOfMonthDate (date): 
"Always choose the first day of the month" 
def _ new__(cls, year, month, day): 
return super().__new__(cls, year, month, 1) 


class NamedInt (int): 
"Allow text names for some numbers" 


xlat = {'zero': 0, 'one': 1, 'ten': 10} 
def _new_ (cls, value): 
value = cls.xlat.get (value, value) 


return super().__new__(cls, value) 


class TitleStr (str): 
"Convert str to name suitable for a URL path" 
def __new__(cls, s): 


s = s.lower().replace(' ', '-') 
s = ''.join([c for c in s if c.isalnum() or c == '-']) 
return super().__new__(cls, s) 


The classes can be used like this: 


>>> FirstOfMonthDate (2012, 2, 14) 
FirstOfMonthDate (2012, 2, 1) 

>>> NamedInt ('ten') 

10 

>>> NamedInt (20) 

20 

>>> TitleStr('Blog: Why Python Rocks') 
"blog-why-python-rocks' 
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2.6.16 How do | cache method calls? 


The two principal tools for caching methods are functools.cached_property() and functools. 
lru_cache (). The former stores results at the instance level and the latter at the class level. 


The cached_property approach only works with methods that do not take any arguments. It does not create a reference 
to the instance. The cached method result will be kept only as long as the instance is alive. 


The advantage is that when an instance is no longer used, the cached method result will be released right away. The 
disadvantage is that 1f instances accumulate, so too will the accumulated method results. They can grow without 
bound. 


The /ru_cache approach works with methods that have hashable arguments. It creates a reference to the instance 
unless special efforts are made to pass in weak references. 


The advantage of the least recently used algorithm is that the cache is bounded by the specified maxsize. The disad- 
vantage is that instances are kept alive until they age out of the cache or until the cache is cleared. 


This example shows the various techniques: 


class Weather: 
"Lookup weather information on a government website" 


def __ init_ (self, station_id): 
self. _station_id = station_id 
# The _station_id is private and immutable 


def current_temperature (self): 
"Latest hourly observation" 
# Do not cache this because old results 
# can be out of date. 


@cached_property 

def location(self): 
"Return the longitude/latitude coordinates of the station" 
# Result only depends on the station_id 


@lru_cache (maxsize=20) 

def historic_rainfall(self, date, units='mm'): 
"Rainfall on a given date" 
# Depends on the station_id, date, and units. 


The above example assumes that the station_id never changes. If the relevant instance attributes are mutable, the 
cached_property approach can’t be made to work because it cannot detect changes to the attributes. 


To make the /ru_cache approach work when the station_id is mutable, the class needs to define the __eq__ () and 
__hash__ () methods so that the cache can detect relevant attribute updates: 


class Weather: 
"Example with a mutable station identifier" 


def __ init_ (self, station_id): 
self.station_id = station_id 


def change_station(self, station_id): 
self.station_id = station_id 


def __eg_ (self, other): 
return self.station_id == other.station_id 


def _ hash_ (self): 
return hash (self.station_id) 


@lru_cache (maxsize=20) 
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def historic_rainfall (self, date, units='cm'): 
"Rainfall on a given date' 
# Depends on the station_id, date, and units. 


2.7 Modules 


2.7.1 How do | create a .pyc file? 


When a module is imported for the first time (or when the source file has changed since the current compiled file 
was created) a . pyc file containing the compiled code should be created ina __pycache__ subdirectory of the 
directory containing the . py file. The . pyc file will have a filename that starts with the same name as the . py file, 
and ends with .pyc, with a middle component that depends on the particular python binary that created it. (See 
PEP 3147 for details.) 


One reason that a . pyc file may not be created is a permissions problem with the directory containing the source 
file, meaning that the __pycache__ subdirectory cannot be created. This can happen, for example, if you develop 
as one user but run as another, such as if you are testing with a web server. 


Unless the PYTHONDONTWRITEBYTECODE environment variable is set, creation of a .pyc file is automatic if you're 
importing a module and Python has the ability (permissions, free space, etc...) to create a__ pycache__ subdi- 
rectory and write the compiled module to that subdirectory. 


Running Python on a top level script is not considered an import and no . pyc will be created. For example, if you 
have a top-level module foo. py that imports another module xyz . py, when you run foo (by typing python 
foo.py as a shell command), a . pyc will be created for xyz because xyz is imported, but no . pyc file will be 
created for foo since foo. py isn’t being imported. 


If you need to create a . pyc file for foo — that is, to create a . pyc file for a module that is not imported — you can, 
using the py_compile and compileall modules. 


The py_compi le module can manually compile any module. One way is to use the compile () function in that 
module interactively: 


>>> import py_compile 
>>> py_compile.compile('foo.py') 


This will write the .pyc toa ___pycache__ subdirectory in the same location as foo. py (or you can override 
that with the optional parameter cfile). 


You can also automatically compile all files in a directory or directories using the compileall module. You can 
do it from the shell prompt by running compileall.py and providing the path of a directory containing Python 
files to compile: 


python -m compileall 


2.7.2 How do | find the current module name? 


A module can find out its own module name by looking at the predefined global variable __ name__. If this has the 
value '__main__ ',the program is running as a script. Many modules that are usually used by importing them also 
provide a command-line interface or a self-test, and only execute this code after checking __ name__: 


def main(): 
print ('Running test...') 
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2.7.3 How can | have modules that mutually import each other? 


Suppose you have the following modules: 


foo.py: 

from bar import bar_var 
foo_var = 1 

bar.py: 


from foo import foo_var 
bar_var = 2 


The problem is that the interpreter will perform the following steps: 
e main imports foo 
e Empty globals for foo are created 
e foo is compiled and starts executing 
e foo imports bar 
e Empty globals for bar are created 
e bar is compiled and starts executing 
e bar imports foo (which is a no-op since there already is a module named foo) 
e The import mechanism tries to read foo_var from foo globals, tosetbar.foo_var = foo.foo_var 


The last step fails, because Python isn’t done with interpreting foo yet and the global symbol dictionary for foo is 
still empty. 


The same thing happens when you use import foo, and then try to access foo. foo_var in global code. 
There are (at least) three possible workarounds for this problem. 


Guido van Rossum recommends avoiding all uses of from <module> import ...,and placing all code inside 
functions. Initializations of global variables and class variables should use constants or built-in functions only. This 
means everything from an imported module is referenced as <module>.<name>. 


Jim Roskind suggests performing steps in the following order in each module: 

e exports (globals, functions, and classes that don't need imported base classes) 

e import statements 

e active code (including globals that are initialized from imported values). 
Van Rossum doesn’t like this approach much because the imports appear in a strange place, but it does work. 
Matthias Urlichs recommends restructuring your code so that the recursive import is not necessary in the first place. 


These solutions are not mutually exclusive. 


2.7.4 _import__(‘x.y.z’) returns <module ‘x’>; how do | get z? 


Consider using the convenience function import_module () from import1ib instead: 


z = importlib.import_module('x.y.z') 
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2.7.5 When I edit an imported module and reimport it, the changes don’t show 
up. Why does this happen? 


For reasons of efficiency as well as consistency, Python only reads the module file on the first time a module is 
imported. If it didn't, in a program consisting of many modules where each one imports the same basic module, the 
basic module would be parsed and re-parsed many times. To force re-reading of a changed module, do this: 


import importlib 
import modname 
importlib.reload (modname) 


Warning: this technique is not 100% fool-proof. In particular, modules containing statements like 


from modname import some_objects 


will continue to work with the old version of the imported objects. If the module contains class definitions, existing 
class instances will not be updated to use the new class definition. This can result in the following paradoxical 
behaviour: 


>>> import importlib 

>>> import cls 

>>> c = cls.C() # Create an instance of C 
>>> importlib.reload(cls) 

<module 'cls' from 'cls.py'> 

>>> isinstance(c, cls.C) # isinstance is false?!? 
False 


The nature of the problem is made clear if you print out the “identity” of the class objects: 


>>> hex(id(c.__class__)) 
'0x7352a0' 

>>> hex(id(cls.C)) 
"0x4198d0' 
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3.1 Why does Python use indentation for grouping of statements? 


Guido van Rossum believes that using indentation for grouping is extremely elegant and contributes a lot to the clarity 
of the average Python program. Most people learn to love this feature after a while. 


Since there are no begin/end brackets there cannot be a disagreement between grouping perceived by the parser and 
the human reader. Occasionally C programmers will encounter a fragment of code like this: 


if (x <= y) 
X++5 
Y >; 
Z++; 


Only the x++ statement is executed if the condition is true, but the indentation leads many to believe otherwise. Even 
experienced C programmers will sometimes stare at it a long time wondering as to why y is being decremented even 
forx > y. 


Because there are no begin/end brackets, Python is much less prone to coding-style conflicts. In C there are many 
different ways to place the braces. After becoming used to reading and writing code using a particular style, it is 
normal to feel somewhat uneasy when reading (or being required to write) in a different one. 


Many coding styles place begin/end brackets on a line by themselves. This makes programs considerably longer and 
wastes valuable screen space, making it harder to get a good overview of a program. Ideally, a function should fit on 
one screen (say, 20-30 lines). 20 lines of Python can do a lot more work than 20 lines of C. This is not solely due to 
the lack of begin/end brackets — the lack of declarations and the high-level data types are also responsible — but the 
indentation-based syntax certainly helps. 


3.2 Why am | getting strange results with simple arithmetic oper- 
ations? 


See the next question. 


3.3 Why are floating-point calculations so inaccurate? 


Users are often surprised by results like this: 


>>> 1.2 = 1,0 
0.19999999999999996 


and think it is a bug in Python. It's not. This has little to do with Python, and much more to do with how the underlying 
platform handles floating-point numbers. 
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The float type in CPython uses a C double for storage. A float object's value is stored in binary floating-point 
with a fixed precision (typically 53 bits) and Python uses C operations, which in turn rely on the hardware implemen- 
tation in the processor, to perform floating-point operations. This means that as far as floating-point operations are 
concerned, Python behaves like many popular languages including C and Java. 


Many numbers that can be written easily in decimal notation cannot be expressed exactly in binary floating-point. 
For example, after: 


>>> x = 1.2 


the value stored for x is a (very good) approximation to the decimal value 1 . 2, but is not exactly equal to it. On a 
typical machine, the actual stored value is: 


1.0011001100110011001100110011001100110011001100110011 (binary) 


which is exactly: 


1.1999999999999999555910790149937383830547332763671875 (decimal) 


The typical precision of 53 bits provides Python floats with 15-16 decimal digits of accuracy. 


For a fuller explanation, please see the floating point arithmetic chapter in the Python tutorial. 


3.4 Why are Python strings immutable? 


There are several advantages. 


One is performance: knowing that a string is immutable means we can allocate space for it at creation time, and the 
storage requirements are fixed and unchanging. This is also one of the reasons for the distinction between tuples and 
lists. 


Another advantage is that strings in Python are considered as “elemental” as numbers. No amount of activity will 
change the value 8 to anything else, and in Python, no amount of activity will change the string “eight” to anything 
else. 


3.5 Why must ‘self’ be used explicitly in method definitions and 
calls? 


The idea was borrowed from Modula-3. It turns out to be very useful, for a variety of reasons. 


First, it’s more obvious that you are using a method or instance attribute instead of a local variable. Reading self .x 
or self .meth () makes it absolutely clear that an instance variable or method is used even if you don’t know the 
class definition by heart. In C++, you can sort of tell by the lack of a local variable declaration (assuming globals are 
rare or easily recognizable) — but in Python, there are no local variable declarations, so you'd have to look up the class 
definition to be sure. Some C++ and Java coding standards call for instance attributes to have an m_ prefix, so this 
explicitness is still useful in those languages, too. 


Second, it means that no special syntax is necessary if you want to explicitly reference or call the method from a 
particular class. In C++, if you want to use a method from a base class which is overridden in a derived class, 
you have to use the : : operator — in Python you can write baseclass.methodname (self, <argument 
list>). This is particularly useful for__init__ () methods, and in general in cases where a derived class method 
wants to extend the base class method of the same name and thus has to call the base class method somehow. 


Finally, for instance variables it solves a syntactic problem with assignment: since local variables in Python are (by 
definition!) those variables to which a value is assigned in a function body (and that aren't explicitly declared global), 
there has to be some way to tell the interpreter that an assignment was meant to assign to an instance variable instead 
of to a local variable, and it should preferably be syntactic (for efficiency reasons). C++ does this through declarations, 
but Python doesn't have declarations and it would be a pity having to introduce them just for this purpose. Using the 
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explicit sel f . var solves this nicely. Similarly, for using instance variables, having to write self . var means that 
references to unqualified names inside a method don't have to search the instance's directories. To put it another way, 
local variables and instance variables live in two different namespaces, and you need to tell Python which namespace 
to use. 


3.6 Why can't | use an assignment in an expression? 


Starting in Python 3.8, you can! 


Assignment expressions using the walrus operator : = assign a variable in an expression: 


while chunk := fp.read(200): 
print (chunk) 


See PEP 572 for more information. 


3.7 Why does Python use methods for some functionality (e.g. 
list.index()) but functions for other (e.g. len(list))? 


As Guido said: 


(a) For some operations, prefix notation just reads better than postfix — prefix (and infix!) operations have 
a long tradition in mathematics which likes notations where the visuals help the mathematician thinking 
about a problem. Compare the easy with which we rewrite a formula like x*(a+b) into x*a + x*b to the 
clumsiness of doing the same thing using a raw OO notation. 


(b) When I read code that says len(x) I know that it is asking for the length of something. This tells 
me two things: the result is an integer, and the argument is some kind of container. To the contrary, 
when I read x.len(), I have to already know that x is some kind of container implementing an interface 
or inheriting from a class that has a standard len(). Witness the confusion we occasionally have when a 
class that is not implementing a mapping has a get() or keys() method, or something that isn’t a file has 
a write() method. 


—https://mail.python.org/pipermail/python-3000/2006- November/004643.html 


3.8 Why is join() a string method instead of a list or tuple method? 


Strings became much more like other standard types starting in Python 1.6, when methods were added which give 
the same functionality that has always been available using the functions of the string module. Most of these new 
methods have been widely accepted, but the one which appears to make some programmers feel uncomfortable is: 


ue "orar, Lat. nar. gr. 116']) 


which gives the result: 


WU o BGs O 


There are two common arguments against this usage. 


The first runs along the lines of: “It looks really ugly using a method of a string literal (string constant)”, to which the 
answer is that it might, but a string literal is just a fixed value. If the methods are to be allowed on names bound to 
strings there is no logical reason to make them unavailable on literals. 


The second objection is typically cast as: “I am really telling a sequence to join its members together with a string 
constant”. Sadly, you aren't. For some reason there seems to be much less difficulty with having split () as a string 
method, since in that case it is easy to see that 
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"Ip 2p Ap By LE" split ("y Y] 


is an instruction to a string literal to return the substrings delimited by the given separator (or, by default, arbitrary 
runs of white space). 


join () isa string method because in using it you are telling the separator string to iterate over a sequence of strings 
and insert itself between adjacent elements. This method can be used with any argument which obeys the rules for 
sequence objects, including any new classes you might define yourself. Similar methods exist for bytes and bytearray 
objects. 


3.9 How fast are exceptions? 


A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive. In 
versions of Python prior to 2.0 1t was common to use this idiom: 


try: 
value = mydict [key] 

except KeyError: 
mydict [key] = getvalue (key) 
value = mydict [key] 


This only made sense when you expected the dict to have the key almost all the time. If that wasn’t the case, you 
coded it like this: 


if key in mydict: 
value = mydict [key] 
else: 
value = mydict[key] = getvalue (key) 


For this specific case, you could also use value = dict.setdefault (key, getvalue (key) ), but only 
if the get value () call is cheap enough because it is evaluated in all cases. 


3.10 Why isn’t there a switch or case statement in Python? 


You can do this easily enough with a sequence of if... elif... elif... else. For literal values, or 
constants within a namespace, you can also use a match ... case statement. 


For cases where you need to choose from a very large number of possibilities, you can create a dictionary mapping 
case values to functions to call. For example: 


functions = ('a': function 1, 
"p's function 2, 
'c': self.method_1) 


func = functions[value] 
func () 


For calling methods on objects, you can simplify yet further by using the get attr () built-in to retrieve methods 
with a particular name: 


class MyVisitor: 
def visit_a(self): 


def dispatch(self, value): 


method_name = 'visit_' + str (value) 
method = getattr(self, method_name) 
method () 
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It’s suggested that you use a prefix for the method names, such as visit _ in this example. Without such a prefix, if 
values are coming from an untrusted source, an attacker would be able to call any method on your object. 


3.11 Can't you emulate threads in the interpreter instead of relying 
on an OS-specific thread implementation? 


Answer 1: Unfortunately, the interpreter pushes at least one C stack frame for each Python stack frame. Also, 
extensions can call back into Python at almost random moments. Therefore, a complete threads implementation 
requires thread support for C. 


Answer 2: Fortunately, there is Stackless Python, which has a completely redesigned interpreter loop that avoids the 
C stack. 


3.12 Why can't lambda expressions contain statements? 


Python lambda expressions cannot contain statements because Python's syntactic framework can't handle statements 
nested inside expressions. However, in Python, this is not a serious problem. Unlike lambda forms in other languages, 
where they add functionality, Python lambdas are only a shorthand notation if you're too lazy to define a function. 


Functions are already first class objects in Python, and can be declared in a local scope. Therefore the only advantage 
of using a lambda instead of a locally defined function is that you don't need to invent a name for the function — 
but that's just a local variable to which the function object (which is exactly the same type of object that a lambda 
expression yields) is assigned! 


3.13 Can Python be compiled to machine code, C or some other 
language? 


Cython compiles a modified version of Python with optional annotations into C extensions. Nuitka is an up-and- 
coming compiler of Python into C++ code, aiming to support the full Python language. 


3.14 How does Python manage memory? 


The details of Python memory management depend on the implementation. The standard implementation of Python, 
CPython, uses reference counting to detect inaccessible objects, and another mechanism to collect reference cycles, 
periodically executing a cycle detection algorithm which looks for inaccessible cycles and deletes the objects involved. 
The gc module provides functions to perform a garbage collection, obtain debugging statistics, and tune the collector’s 
parameters. 


Other implementations (such as Jython or PyPy), however, can rely on a different mechanism such as a full-blown 
garbage collector. This difference can cause some subtle porting problems if your Python code depends on the 
behavior of the reference counting implementation. 


In some Python implementations, the following code (which is fine in CPython) will probably run out of file descrip- 
tors: 


for file in very_long_list_of_files: 
f = open(file) 
c = f.read(1) 


Indeed, using CPython’s reference counting and destructor scheme, each new assignment to f closes the previous file. 
With a traditional GC, however, those file objects will only get collected (and closed) at varying and possibly long 
intervals. 
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If you want to write code that will work with any Python implementation, you should explicitly close the file or use 
the with statement; this will work regardless of memory management scheme: 


for file in very_long_list_of_files: 
with open(file) as f: 
c = f.read(1) 


3.15 Why doesn’t CPython use a more traditional garbage collec- 
tion scheme? 


For one thing, this is not a C standard feature and hence it’s not portable. (Yes, we know about the Boehm GC library. 
It has bits of assembler code for most common platforms, not for all of them, and although it is mostly transparent, 
it isn’t completely transparent; patches are required to get Python to work with it.) 


Traditional GC also becomes a problem when Python is embedded into other applications. While in a standalone 
Python it’s fine to replace the standard malloc() and free() with versions provided by the GC library, an application 
embedding Python may want to have its own substitute for malloc() and free(), and may not want Python’s. Right 
now, CPython works with anything that implements malloc() and free() properly. 


3.16 Why isn’t all memory freed when CPython exits? 


Objects referenced from the global namespaces of Python modules are not always deallocated when Python exits. 
This may happen if there are circular references. There are also certain bits of memory that are allocated by the C 
library that are impossible to free (e.g. a tool like Purify will complain about these). Python is, however, aggressive 
about cleaning up memory on exit and does try to destroy every single object. 


If you want to force Python to delete certain things on deallocation use the atexit module to run a function that 
will force those deletions. 


3.17 Why are there separate tuple and list data types? 


Lists and tuples, while similar in many respects, are generally used in fundamentally different ways. Tuples can be 
thought of as being similar to Pascal records or C structs; they’re small collections of related data which may be of 
different types which are operated on as a group. For example, a Cartesian coordinate is appropriately represented 
as a tuple of two or three numbers. 


Lists, on the other hand, are more like arrays in other languages. They tend to hold a varying number of objects all 
of which have the same type and which are operated on one-by-one. For example, os. listdir('."') returns a 
list of strings representing the files in the current directory. Functions which operate on this output would generally 
not break if you added another file or two to the directory. 


Tuples are immutable, meaning that once a tuple has been created, you can’t replace any of its elements with a new 
value. Lists are mutable, meaning that you can always change a list’s elements. Only immutable elements can be used 
as dictionary keys, and hence only tuples and not lists can be used as keys. 
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3.18 How are lists implemented in CPython? 


CPython's lists are really variable-length arrays, not Lisp-style linked lists. The implementation uses a contiguous 
array of references to other objects, and keeps a pointer to this array and the array's length in a list head structure. 


This makes indexing a list a [i] an operation whose cost is independent of the size of the list or the value of the 
index. 


When items are appended or inserted, the array of references is resized. Some cleverness is applied to improve the 
performance of appending items repeatedly; when the array must be grown, some extra space is allocated so the next 
few times don’t require an actual resize. 


3.19 How are dictionaries implemented in CPython? 


CPython’s dictionaries are implemented as resizable hash tables. Compared to B-trees, this gives better performance 
for lookup (the most common operation by far) under most circumstances, and the implementation is simpler. 


Dictionaries work by computing a hash code for each key stored in the dictionary using the hash () built-in function. 
The hash code varies widely depending on the key and a per-process seed; for example, “Python” could hash to - 
539294296 while “python”, a string that differs by a single bit, could hash to 1142331976. The hash code is then 
used to calculate a location in an internal array where the value will be stored. Assuming that you're storing keys that 
all have different hash values, this means that dictionaries take constant time — O(1), in Big-O notation — to retrieve 
a key. 


3.20 Why must dictionary keys be immutable? 


The hash table implementation of dictionaries uses a hash value calculated from the key value to find the key. If the 
key were a mutable object, its value could change, and thus its hash could also change. But since whoever changes 
the key object can’t tell that it was being used as a dictionary key, it can’t move the entry around in the dictionary. 
Then, when you try to look up the same object in the dictionary it won’t be found because its hash value is different. 
If you tried to look up the old value it wouldn’t be found either, because the value of the object found in that hash bin 
would be different. 


If you want a dictionary indexed with a list, simply convert the list to a tuple first; the function tuple (L) creates a 
tuple with the same entries as the list L. Tuples are immutable and can therefore be used as dictionary keys. 


Some unacceptable solutions that have been proposed: 


e Hash lists by their address (object ID). This doesn’t work because if you construct a new list with the same 
value it won’t be found; e.g.: 


mydict = {[1, 2]: '12') 
print (mydict[[1, 2]]) 


would raise a KeyError exception because the id of the [1, 2] used in the second line differs from that 
in the first line. In other words, dictionary keys should be compared using ==, not using is. 


Make a copy when using a list as a key. This doesn’t work because the list, being a mutable object, could 
contain a reference to itself, and then the copying code would run into an infinite loop. 


Allow lists as keys but tell the user not to modify them. This would allow a class of hard-to-track bugs in pro- 
grams when you forgot or modified a list by accident. It also invalidates an important invariant of dictionaries: 
every value in d. keys () is usable as a key of the dictionary. 


Mark lists as read-only once they are used as a dictionary key. The problem is that it’s not just the top-level 
object that could change its value; you could use a tuple containing a list as a key. Entering anything as a key into 
a dictionary would require marking all objects reachable from there as read-only — and again, self-referential 
objects could cause an infinite loop. 
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There is a trick to get around this if you need to, but use it at your own risk: You can wrap a mutable structure inside 
a class instance which has botha__eq__()anda__hash__ () method. You must then make sure that the hash 
value for all such wrapper objects that reside in a dictionary (or other hash based structure), remain fixed while the 
object is in the dictionary (or other structure). 


class ListWrapper: 
def _ _ init_ (self, the_list): 
self.the_list = the_list 


def _ eg_ (self, other): 
return self.the_list == other.the_list 


def __ hash_ (self): 
1 = self.the_list 
result = 98767 - len(1)*555 
for i, l in enumerate (1): 


try: 

result = result + (hash(el) % 9999999) * 1001 + i 
except Exception: 

result = (result % 7777777) + i * 333 


return result 


Note that the hash computation is complicated by the possibility that some members of the list may be unhashable 
and also by the possibility of arithmetic overflow. 


Furthermore it must always be the case thatifo1 == o2(ieo1l.__eq__(02) is True)thenhash (01) == 
hash (02) (ie, o1.__hash__() == o2.__hash__ ()), regardless of whether the object is in a dictionary 
or not. If you fail to meet these restrictions dictionaries and other hash based structures will misbehave. 


In the case of ListWrapper, whenever the wrapper object is in a dictionary the wrapped list must not change to avoid 
anomalies. Don't do this unless you are prepared to think hard about the requirements and the consequences of not 
meeting them correctly. Consider yourself warned. 


3.21 Why doesn’t list.sort() return the sorted list? 


In situations where performance matters, making a copy of the list just to sort 1t would be wasteful. Therefore, 
list.sort () sorts the list in place. In order to remind you of that fact, it does not return the sorted list. This 
way, you won't be fooled into accidentally overwriting a list when you need a sorted copy but also need to keep the 
unsorted version around. 


If you want to return a new list, use the built-in sorted () function instead. This function creates a new list from 
a provided iterable, sorts it and returns it. For example, here’s how to iterate over the keys of a dictionary in sorted 
order: 


for key in sorted(mydict): 
# do whatever with mydict [key]... 


3.22 How do you specify and enforce an interface spec in Python? 


An interface specification for a module as provided by languages such as C++ and Java describes the prototypes for 
the methods and functions of the module. Many feel that compile-time enforcement of interface specifications helps 
in the construction of large programs. 


Python 2.6 adds an abc module that lets you define Abstract Base Classes (ABCs). You can then use 
isinstance() and issubclass() to check whether an instance or a class implements a particular ABC. 
The collections.abc module defines a set of useful ABCs such as Iterable, Container, and 
MutableMapping. 
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For Python, many of the advantages of interface specifications can be obtained by an appropriate test discipline for 
components. 


A good test suite for a module can both provide a regression test and serve as a module interface specification and a 
set of examples. Many Python modules can be run as a script to provide a simple “self test.” Even modules which use 
complex external interfaces can often be tested in isolation using trivial “stub” emulations of the external interface. 
The doctest and unittest modules or third-party test frameworks can be used to construct exhaustive test 
suites that exercise every line of code in a module. 


An appropriate testing discipline can help build large complex applications in Python as well as having interface 
specifications would. In fact, it can be better because an interface specification cannot test certain properties of a 
program. For example, the append () method is expected to add new elements to the end of some internal list; an 
interface specification cannot test that your append () implementation will actually do this correctly, but it’s trivial 
to check this property in a test suite. 


Writing test suites is very helpful, and you might want to design your code to make it easily tested. One increasingly 
popular technique, test-driven development, calls for writing parts of the test suite first, before you write any of the 
actual code. Of course Python allows you to be sloppy and not write test cases at all. 


3.23 Why is there no goto? 


In the 1970s people realized that unrestricted goto could lead to messy “spaghetti” code that was hard to understand 
and revise. In a high-level language, it is also unneeded as long as there are ways to branch (in Python, with if 
statements and or, and, and i f-e1 se expressions) and loop (with while and for statements, possibly containing 
continue and break). 


One can also use exceptions to provide a “structured goto” that works even across function calls. Many feel that 
exceptions can conveniently emulate all reasonable uses of the “go” or “goto” constructs of C, Fortran, and other 
languages. For example: 


class label (Exception): pass # declare a label 


try: 

if condition: raise label() # goto label 
except label: # where to goto 

pass 


This doesn't allow you to jump into the middle of a loop, but that's usually considered an abuse of goto anyway. Use 
sparingly. 


3.24 Why can't raw strings (r-strings) end with a backslash? 


More precisely, they can't end with an odd number of backslashes: the unpaired backslash at the end escapes the 
closing quote character, leaving an unterminated string. 


Raw strings were designed to ease creating input for processors (chiefly regular expression engines) that want to do 
their own backslash escape processing. Such processors consider an unmatched trailing backslash to be an error 
anyway, so raw strings disallow that. In return, they allow you to pass on the string quote character by escaping it 
with a backslash. These rules work well when r-strings are used for their intended purpose. 


If you're trying to build Windows pathnames, note that all Windows system calls accept forward slashes too: 


f = open("/mydir/file.txt") # works fine! 


If you're trying to build a pathname for a DOS command, try e.g. one of 
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dir = ENthisXNisimyldos die” "NN" 
dir = r"\this\is\my\dos\dir\ "[:-1] 
dir = "\\this\\is\\my\\dos\\dir\\" 


3.25 Why doesn’t Python have a “with” statement for attribute as- 
signments? 


Python has a ‘with’ statement that wraps the execution of a block, calling code on the entrance and exit from the 
block. Some languages have a construct that looks like this: 


with obj: 
a= 1 # equivalent to obj.a = 1 
total = total + 1 # obj.total = obj.total + 1 


In Python, such a construct would be ambiguous. 


Other languages, such as Object Pascal, Delphi, and C++, use static types, so it’s possible to know, in an unambiguous 
way, what member is being assigned to. This is the main point of static typing — the compiler always knows the scope 
of every variable at compile time. 


Python uses dynamic types. It is impossible to know in advance which attribute will be referenced at runtime. Member 
attributes may be added or removed from objects on the fly. This makes it impossible to know, from a simple reading, 
what attribute is being referenced: a local one, a global one, or a member attribute? 


For instance, take the following incomplete snippet: 


def foo(a): 
with a: 
print (x) 


The snippet assumes that “a” must have a member attribute called “x”. However, there is nothing in Python that tells 
the interpreter this. What should happen if “a” is, let us say, an integer? If there is a global variable named “x”, will 
it be used inside the with block? As you see, the dynamic nature of Python makes such choices much harder. 


The primary benefit of “with” and similar language features (reduction of code volume) can, however, easily be 
achieved in Python by assignment. Instead of: 


function (args) .mydict [index] [index].a 
function(args) .mydict [index] [index].b = 42 
function(args) .mydict [index] [index].c = 63 


write this: 


ref = function(args) .mydict [index] [index] 
ref.a = 21 
ref.b = 42 
ref.c = 63 


This also has the side-effect of increasing execution speed because name bindings are resolved at run-time in Python, 
and the second version only needs to perform the resolution once. 


48 Chapter 3. Design and History FAQ 


Python Frequently Asked Questions, Release 3.11.1 


3.26 Why don't generators support the with statement? 


For technical reasons, a generator used directly as a context manager would not work correctly. When, as is most 
common, a generator is used as an iterator run to completion, no closing is needed. When it is, wrap it as “con- 
textlib.closing(generator)” in the ‘with’ statement. 


3.27 Why are colons required for the if/while/def/class state- 
ments? 


The colon is required primarily to enhance readability (one of the results of the experimental ABC language). Con- 
sider this: 


if a == b 
print (a) 


versus 


if a == 
print (a) 


Notice how the second one is slightly easier to read. Notice further how a colon sets off the example in this FAQ 
answer; it's a standard usage in English. 


Another minor reason is that the colon makes it easier for editors with syntax highlighting; they can look for colons to 
decide when indentation needs to be increased instead of having to do a more elaborate parsing of the program text. 


3.28 Why does Python allow commas at the end of lists and tu- 
ples? 


Python lets you add a trailing comma at the end of lists, tuples, and dictionaries: 


r 
"Bme [67 TI # last trailing comma is optional but good style 


There are several reasons to allow this. 


When you have a literal value for a list, tuple, or dictionary spread across multiple lines, it’s easier to add more 
elements because you don’t have to remember to add a comma to the previous line. The lines can also be reordered 
without creating a syntax error. 


Accidentally omitting the comma can lead to errors that are hard to diagnose. For example: 


x = [ 
"fee", 
"fie" 
"toa", 
"Fun? 


] 


This list looks like it has four elements, but it actually contains three: “fee”, “fiefoo” and “fum”. Always adding the 
comma avoids this source of error. 


Allowing the trailing comma may also make programmatic code generation easier. 
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CHAPTER 
FOUR 


LIBRARY AND EXTENSION FAQ 


4.1 General Library Questions 


4.1.1 How do | find a module or application to perform task X? 
Check the Library Reference to see if there's a relevant standard library module. (Eventually you'll learn what's in 
the standard library and will be able to skip this step.) 


For third-party packages, search the Python Package Index or try Google or another web search engine. Searching 
for “Python” plus a keyword or two for your topic of interest will usually find something helpful. 


4.1.2 Where is the math.py (socket.py, regex.py, etc.) source file? 


If you can’t find a source file for a module it may be a built-in or dynamically loaded module implemented in C, C++ or 
other compiled language. In this case you may not have the source file or it may be something like mat hmodule.c, 
somewhere in a C source directory (not on the Python Path). 


There are (at least) three kinds of modules in Python: 
1) modules written in Python (.py); 
2) modules written in C and dynamically loaded (.dll, .pyd, .so, .sl, etc); 


3) modules written in C and linked with the interpreter; to get a list of these, type: 


import sys 
print (sys.builtin_module_names) 


4.1.3 How do | make a Python script executable on Unix? 


You need to do two things: the script file’s mode must be executable and the first line must begin with + ! followed 
by the path of the Python interpreter. 


The first is done by executing chmod +x scriptfile or perhaps chmod 755 scriptfile. 


The second can be done in a number of ways. The most straightforward way is to write 


#!/usr/local/bin/python 


as the very first line of your file, using the pathname for where the Python interpreter is installed on your platform. 


If you would like the script to be independent of where the Python interpreter lives, you can use the env program. 
Almost all Unix variants support the following, assuming the Python interpreter is in a directory on the user’s PATH: 


#!/usr/bin/env python 
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Dont do this for CGI scripts. The PATH variable for CGI scripts is often very minimal, so you need to use the actual 
absolute pathname of the interpreter. 


Occasionally, a user’s environment is so full that the /usr/bin/env program fails; or there’s no env program at 
all. In that case, you can try the following hack (due to Alex Rezinsky): 


#! /bin/sh 


"nunen 


exec python $0 S{1+"S@"} 


nnn 


The minor disadvantage is that this defines the scripts __doc__ string. However, you can fix that by adding 


doc = wv... Whatever, sn TM 


4.1.4 Is there a curses/termcap package for Python? 


For Unix variants: The standard Python source distribution comes with a curses module in the Modules subdirectory, 
though it’s not compiled by default. (Note that this is not available in the Windows distribution — there is no curses 
module for Windows.) 


The curses module supports basic curses features as well as many additional functions from ncurses and SYSV 
curses such as colour, alternative character set support, pads, and mouse support. This means the module isn’t com- 
patible with operating systems that only have BSD curses, but there don’t seem to be any currently maintained OSes 
that fall into this category. 


4.1.5 Is there an equivalent to C’s onexit() in Python? 


The atexit module provides a register function that is similar to C’s onexit (). 


4.1.6 Why don’t my signal handlers work? 


The most common problem is that the signal handler is declared with the wrong argument list. It is called as 


handler(signum, frame) 


so it should be declared with two parameters: 


def handler(signum, frame): 


4.2 Common tasks 


4.2.1 How do | test a Python program or component? 

Python comes with two testing frameworks. The doctest module finds examples in the docstrings for a module 
and runs them, comparing the output with the expected output given in the docstring. 

The unittest module is a fancier testing framework modelled on Java and Smalltalk testing frameworks. 


To make testing easier, you should use good modular design in your program. Your program should have almost all 
functionality encapsulated in either functions or class methods — and this sometimes has the surprising and delightful 
effect of making the program run faster (because local variable accesses are faster than global accesses). Furthermore 
the program should avoid depending on mutating global variables, since this makes testing much more difficult to do. 


The “global main logic” of your program may be as simple as 
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if name == "__main_ ": 


main_logic () 


at the bottom of the main module of your program. 


Once your program is organized as a tractable collection of function and class behaviours, you should write test 
functions that exercise the behaviours. A test suite that automates a sequence of tests can be associated with each 
module. This sounds like a lot of work, but since Python is so terse and flexible it’s surprisingly easy. You can make 
coding much more pleasant and fun by writing your test functions in parallel with the “production code”, since this 
makes it easy to find bugs and even design flaws earlier. 


“Support modules” that are not intended to be the main module of a program may include a self-test of the module. 


if name == "__main_ ": 
self_test () 


Even programs that interact with complex external interfaces may be tested when the external interfaces are unavail- 
able by using “fake” interfaces implemented in Python. 


4.2.2 How do! create documentation from doc strings? 


The pydoc module can create HTML from the doc strings in your Python source code. An alternative for creating 
API documentation purely from docstrings is epydoc. Sphinx can also include docstring content. 


4.2.3 How do | get a single keypress at a time? 


For Unix variants there are several solutions. It’s straightforward to do this using curses, but curses is a fairly large 
module to learn. 


4.3 Threads 


4.3.1 How do | program using threads? 


Be sure to use the threading module and not the _thread module. The threading module builds convenient 
abstractions on top of the low-level primitives provided by the _thread module. 


4.3.2 None of my threads seem to run: why? 


As soon as the main thread exits, all threads are killed. Your main thread is running too quickly, giving the threads 
no time to do any work. 


A simple fix is to add a sleep to the end of the program that's long enough for all the threads to finish: 


import threading, time 


def thread_task (name, n): 
for i in range(n): 
print (name, 1) 


for i in range(10): 
T = threading.Thread (target=thread_task, args=(str(i), i)) 
T.start () 


time.sleep(10) # <---------------------------! 
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But now (on many platforms) the threads don't run in parallel, but appear to run sequentially, one at a time! The 
reason is that the OS thread scheduler doesn’t start a new thread until the previous thread is blocked. 


A simple fix is to add a tiny sleep to the start of the run function: 


def thread_task (name, n): 
time.sleep(0.001) # <--------------------! 
for i in range(n): 
print (name, i) 


for i in range(10): 
T = threading.Thread(target=thread_task, args=(str(i), i)) 
T.start () 


time.sleep (10) 


Instead of trying to guess a good delay value for time . sleep (), it’s better to use some kind of semaphore mech- 
anism. One idea is to use the queue module to create a queue object, let each thread append a token to the queue 
when it finishes, and let the main thread read as many tokens from the queue as there are threads. 


4.3.3 How do I parcel out work among a bunch of worker threads? 


The easiest way is to use the concurrent. futures module, especially the ThreadPoolExecutor class. 


Or, if you want fine control over the dispatching algorithm, you can write your own logic manually. Use the queue 
module to create a queue containing a list of jobs. The Queue class maintains a list of objects and has a . put (obj) 
method that adds items to the queue and a . get () method to return them. The class will take care of the locking 
necessary to ensure that each job is handed out exactly once. 


Here’s a trivial example: 


import threading, queue, time 


# The worker thread gets jobs off the queue. When the queue is empty, it 
# assumes there will be no more work and exits. 
# (Realistically workers will run until terminated.) 
def worker(): 
print ('Running worker') 
time.sleep (0.1) 
while True: 
try: 
arg = q.get (block=False) 
except queue.Empty: 
print ('Worker', threading.current_thread(), nd=' ') 


print ('queue empty') 
break 
else: 
print ('Worker', threading.current_thread(), end=' ') 
print ("running with argument', arg) 
time.sleep (0.5) 


# Create queue 
q = queue. Queue () 


# Start a pool of 5 workers 

for i in range(5): 
t = threading.Thread(target=worker, name='worker %31' $ (i+1)) 
t.start () 


# Begin adding work to the queue 
for i in range(50): 
q-put (i) 


(continues on next page) 
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(continued from previous page) 


# Give threads time to run 
print ('Main thread sleeping') 
time.sleep (5) 


When run, this will produce the following output: 


Running worker 
Running worker 
Running worker 
Running worker 
Running worker 
Main thread sleeping 
Worker <Thread (worker 
Worker <Thread (worker 
Worker <Thread (worker 
Worker <Thread (worker 
( 
( 


started 130283832797456 
started 130283824404752 
started 130283816012048 
started 130283807619344 
started 130283799226640 
started 130283832797456 


running with argument 
running with argument 
running with argument 
running with argument 
running with argument 
running with argument 


~ 


~ 


~ 


Worker <Thread (worker 
Worker <Thread (worker 


`~ 


en bUr- 
~ 


~ 


OP WDNR OO 


)> 
)> 
)> 
)> 
)> 
)> 


Consult the module’s documentation for more details; the Queue class provides a featureful interface. 


4.3.4 What kinds of global value mutation are thread-safe? 


A global interpreter lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In 
general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be 
set via sys. setswitchinterval (). Each bytecode instruction and therefore all the C implementation code 
reached from each instruction is therefore atomic from the point of view of a Python program. 


In theory, this means an exact accounting requires an exact understanding of the PVM bytecode implementation. In 
practice, it means that operations on shared variables of built-in data types (ints, lists, dicts, etc) that “look atomic” 
really are. 


For example, the following operations are all atomic (L, L1, L2 are lists, D, D1, D2 are dicts, x, y are objects, i, j are 
ints): 


L. append (x) 
L1.extend(L2) 
Xx [il 
x = L.pop() 
DL1[i:3] = L2 
L.sort () 

x = y 
x.field = y 
D[x] = y 
D1.update (D2) 
D. keys () 


These aren't: 


i = itt 
L.append(L[-1]) 
L[i] = L[j] 
D[x] = D[x] + 1 


Operations that replace other objects may invoke those other objects’ ___de1__() method when their reference 
count reaches zero, and that can affect things. This is especially true for the mass updates to dictionaries and lists. 
When in doubt, use a mutex! 


4.3. Threads 55 


Python Frequently Asked Questions, Release 3.11.1 


4.3.5 Can't we get rid of the Global Interpreter Lock? 


The global interpreter lock (GIL) is often seen as a hindrance to Python’s deployment on high-end multiprocessor 
server machines, because a multi-threaded Python program effectively only uses one CPU, due to the insistence that 
(almost) all Python code can only run while the GIL is held. 


Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive patch set (the “free threading” 
patches) that removed the GIL and replaced it with fine-grained locking. Adam Olsen recently did a similar ex- 
periment in his python-safethread project. Unfortunately, both experiments exhibited a sharp drop in single-thread 
performance (at least 30% slower), due to the amount of fine-grained locking necessary to compensate for the removal 
of the GIL. 


This doesn’t mean that you can’t make good use of Python on multi-CPU machines! You just have to be creative 
with dividing the work up between multiple processes rather than multiple threads. The ProcessPoolExecutor 
class in the new concurrent. futures module provides an easy way of doing so; the multiprocessing 
module provides a lower-level API in case you want more control over dispatching of tasks. 


Judicious use of C extensions will also help; if you use a C extension to perform a time-consuming task, the extension 
can release the GIL while the thread of execution is in the C code and allow other threads to get some work done. 
Some standard library modules such as zlib and hash1ib already do this. 


It has been suggested that the GIL should be a per-interpreter-state lock rather than truly global; interpreters then 
wouldn’t be able to share objects. Unfortunately, this isn’t likely to happen either. It would be a tremendous amount 
of work, because many object implementations currently have global state. For example, small integers and short 
strings are cached; these caches would have to be moved to the interpreter state. Other object types have their own 
free list; these free lists would have to be moved to the interpreter state. And so on. 


And I doubt that it can even be done in finite time, because the same problem exists for 3rd party extensions. It is 
likely that 3rd party extensions are being written at a faster rate than you can convert them to store all their global 
state in the interpreter state. 


And finally, once you have multiple interpreters not sharing any state, what have you gained over running each inter- 
preter in a separate process? 


4.4 Input and Output 


4.4.1 How do | delete a file? (And other file questions...) 


Use os. remove (filename) or os.unlink (filename); for documentation, see the os module. The two 
functions are identical; unlink () is simply the name of the Unix system call for this function. 


To remove a directory, use os. rmdir (); use os.mkdir () to create one. os.makedirs (path) will cre- 
ate any intermediate directories in path that don't exist. os.removedirs (path) will remove intermediate 
directories as long as they're empty; if you want to delete an entire directory tree and its contents, use shutil. 
rmtree (). 


To rename a file, use os. rename (old_path, new_path). 


To truncate a file, open it using £ = open (filename, "rb+"), and use f.truncate (offset); offset 
defaults to the current seek position. There's also os. ftruncate (fd, offset) for files opened with os. 
open (), where fd is the file descriptor (a small integer). 


The shut il module also contains a number of functions to work on files including copyfile(), copytree(), 
and rmt ree (). 
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4.4.2 How do | copy a file? 


The shut i1 module contains a copyfile () function. Note that on Windows NTFS volumes, it does not copy 
alternate data streams nor resource forks on macOS HFS+ volumes, though both are now rarely used. It also doesn't 
copy file permissions and metadata, though using shut il.copy2 () instead will preserve most (though not all) of 
it. 


4.4.3 How do | read (or write) binary data? 


To read or write complex binary data formats, it’s best to use the st ruct module. It allows you to take a string 
containing binary data (usually numbers) and convert it to Python objects; and vice versa. 


For example, the following code reads two 2-byte integers and one 4-byte integer in big-endian format from a file: 


import struct 


with open(filename, "rb") as f: 
s = f.read(8) 
X, Y, Zz = struct.unpack(">hh1", s) 


The ‘>’ in the format string forces big-endian data; the letter ‘h’ reads one “short integer” (2 bytes), and T reads one 
“long integer” (4 bytes) from the string. 


For data that is more regular (e.g. a homogeneous list of ints or floats), you can also use the array module. 


Note: To read and write binary data, it is mandatory to open the file in binary mode (here, passing "rb" to 
open ()). If you use "r" instead (the default), the file will be open in text mode and f . read () will return str 
objects rather than bytes objects. 


4.4.4 | can’t seem to use os.read() on a pipe created with os.popen(); why? 


os.read () is a low-level function which takes a file descriptor, a small integer representing the opened file. os. 
popen () creates a high-level file object, the same type returned by the built-in open () function. Thus, to read n 
bytes from a pipe p created with os . popen (), you need to use p. read (n). 


4.4.5 How do I access the serial (RS232) port? 


For Win32, OSX, Linux, BSD, Jython, IronPython: 
https://pypi.org/project/pyserial/ 
For Unix, see a Usenet post by Mitch Chapman: 


https://groups.google.com/groups?selm=34A04430.CF9 @ohioee.com 


4.4.6 Why doesn’t closing sys.stdout (stdin, stderr) really close it? 


Python file objects are a high-level layer of abstraction on low-level C file descriptors. 


For most file objects you create in Python via the built-in open () function, f.close() marks the Python file 
object as being closed from Python’s point of view, and also arranges to close the underlying C file descriptor. This 
also happens automatically in £”s destructor, when f becomes garbage. 


But stdin, stdout and stderr are treated specially by Python, because of the special status also given to them by 
C. Running sys.stdout.close() marks the Python-level file object as being closed, but does not close the 
associated C file descriptor. 


4.4. Input and Output 57 


Python Frequently Asked Questions, Release 3.11.1 


To close the underlying C file descriptor for one of these three, you should first be sure that's what you really want to 
do (e.g., you may confuse extension modules trying to do I/O). If itis, use os. close (): 


os.close(stdin.fileno()) 
os.close(stdout.fileno()) 
os.close(stderr.fileno()) 


Or you can use the numeric constants 0, 1 and 2, respectively. 


4.5 Network/Internet Programming 


4.5.1 What WWW tools are there for Python? 


See the chapters titled internet and netdata in the Library Reference Manual. Python has many modules that will help 
you build server-side and client-side web systems. 


A summary of available frameworks is maintained by Paul Boddie at https://wiki.python.org/moin/ 
WebProgramming. 


Cameron Laird maintains a useful set of pages about Python web technologies at https://web.archive.org/web/ 
20210224183619/http://phaseit.net/claird/comp.lang.python/web_python. 


4.5.2 How can | mimic CGI form submission (METHOD=POST)? 


I would like to retrieve web pages that are the result of POSTing a form. Is there existing code that would let me do 
this easily? 


Yes. Here's a simple example that uses urllib. request: 


#!/usr/local/bin/python 
import urllib.request 


# build the query string 
gs = "First=JosephinesMI=Q8Last=Public" 


# connect and send the server a path 
req = urllib.request.urlopen('http://www.some-server.out-there' 
'/cgi-bin/some-cgi-script', data=qs) 
with req: 
msg, hdrs = req.read(), req.info() 


Note that in general for percent-encoded POST operations, query strings must be quoted using urllib.parse. 
urlencode (). For example, to send name=Guy Steele, Jr.: 


>>> import urllib.parse 
>>> urllib.parse.urlencode({'name': 'Guy Steele, Jr.')) 
'name=Guy+Steele$2C+Jr.' 


See also: 


urllib-howto for extensive examples. 
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4.5.3 What module should | use to help with generating HTML? 


You can find a collection of useful links on the Web Programming wiki page. 


4.5.4 How do | send mail from a Python script? 


Use the standard library module smtplib. 


Here's a very simple interactive mail sender that uses it. This method will work on any host that supports an SMTP 
listener. 


import sys, smtplib 


fromaddr = input ("From: ") 
toaddrs = input("To: ").split(',') 
print ("Enter message, end with ”D:") 
msg = m1 
while True: 

line = sys.stdin.readline() 

if not line: 

break 


msg += line 


# The actual mail send 

server = smtplib.SMTP('localhost') 
server.sendmail(fromaddr, toaddrs, msg) 
server.quit () 


A Unix-only alternative uses sendmail. The location of the sendmail program varies between systems; sometimes it 
is /usr/lib/sendmail, sometimes /usr/sbin/sendmail. The sendmail manual page will help you out. 
Here’s some sample code: 


import os 


ENDMAIL = "/usr/sbin/sendmail" # sendmail location 
= Os.popen("%3s -t -i" % SENDMAIL, "w") 

.write("To: receiver@example.com\n") 

"Subject: test\n") 


-write("\n") # blank line separating headers from body 


5 

p 

p 
p.write( 

P ( 

p.write ("Some text\n") 
P ( 


.write ("some more text\n") 
sts = p.close() 
if sts != 0: 


print ("Sendmail exit status", sts) 


4.5.5 How do I avoid blocking in the connect() method of a socket? 


The select module is commonly used to help with asynchronous I/O on sockets. 


To prevent the TCP connect from blocking, you can set the socket to non-blocking mode. Then when you do the 
socket .connect (), you will either connect immediately (unlikely) or get an exception that contains the error 
number as .errno. errno.EINPROGRESS indicates that the connection is in progress, but hasn't finished yet. 
Different OSes will return different values, so you're going to have to check what's returned on your system. 


You can use the socket .connect_ex () method to avoid creating an exception. It will just return the errno 
value. To poll, you can call socket . connect_ex () again later — 0 or errno. EI SCONN indicate that you're 
connected — or you can pass this socket to select . select () to check if it’s writable. 


Note: The asyncio module provides a general purpose single-threaded and concurrent asynchronous library, 
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which can be used for writing non-blocking network code. The third-party Twisted library is a popular and feature- 
rich alternative. 


4.6 Databases 


4.6.1 Are there any interfaces to database packages in Python? 


Yes. 


Interfaces to disk-based hashes such as DBM and GDBM are also included with standard Python. There is also the 
sqlite3 module, which provides a lightweight disk-based relational database. 


Support for most relational databases is available. See the DatabaseProgramming wiki page for details. 


4.6.2 How do you implement persistent objects in Python? 


The pickle library module solves this in a very general way (though you still can’t store things like open files, 
sockets or windows), and the she 1 ve library module uses pickle and (g)dbm to create persistent mappings containing 
arbitrary Python objects. 


4.7 Mathematics and Numerics 


4.7.1 How do | generate random numbers in Python? 


The standard module random implements a random number generator. Usage is simple: 


import random 
random. random () 


This returns a random floating point number in the range [0, 1). 
There are also many other specialized generators in this module, such as: 

e randrange (a, b) chooses an integer in the range [a, b). 

e uniform(a, b) chooses a floating point number in the range [a, b). 

e normalvariate (mean, sdev) samples the normal (Gaussian) distribution. 
Some higher-level functions operate on sequences directly, such as: 

e choice (S) chooses a random element from a given sequence. 

e shuffle (L) shuffles a list in-place, i.e. permutes it randomly. 


There’s also a Random class you can instantiate to create independent multiple random number generators. 
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5.1 Can | create my own functions in C? 


Yes, you can create built-in modules containing functions, variables, exceptions and even new types in C. This is 
explained in the document extending-index. 


Most intermediate or advanced Python books will also cover this topic. 


5.2 Can | create my own functions in C++? 


Yes, using the C compatibility features found in C++. Place extern "C" { ... } around the Python include 
files and put extern "C" before each function that is going to be called by the Python interpreter. Global or static 
C++ objects with constructors are probably not a good idea. 


5.3 Writing C is hard; are there any alternatives? 


There are a number of alternatives to writing your own C extensions, depending on what you're trying to do. 


Cython and its relative Pyrex are compilers that accept a slightly modified form of Python and generate the cor- 
responding C code. Cython and Pyrex make it possible to write an extension without having to learn Python’s C 
API. 


If you need to interface to some C or C++ library for which no Python extension currently exists, you can try wrapping 
the library’s data types and functions with a tool such as SWIG. SIP, CXX Boost, or Weave are also alternatives for 
wrapping C++ libraries. 


5.4 How can | execute arbitrary Python statements from C? 


The highest-level function to do this is PyRun_SimpleString() which takes a single string argument to be 
executed in the context of the module __main__ and returns O for success and —1 when an exception oc- 
curred (including SyntaxError). If you want more control, use PyRun_String(); see the source for 
PyRun_SimpleString() inPython/pythonrun.c. 
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5.5 How can I evaluate an arbitrary Python expression from C? 


Call the function PyRun_St ring () from the previous question with the start symbol Py_eval_input; it parses 
an expression, evaluates it and returns its value. 


5.6 How do | extract C values from a Python object? 


That depends on the object's type. If it’s a tuple, PyTuple_Size () returns its length and PyTuple_GetItenm() 
returns the item at a specified index. Lists have similar functions, PyListSize() and PyList_GetItem(). 


For bytes, PyBytes_Size () returns its length and PyBytes_AsStringAndSize () provides a pointer to its 
value and its length. Note that Python bytes objects may contain null bytes so C's strlen () should not be used. 


To test the type of an object, first make sure it isn't NULL, and then use PyBytes_Check(), 
PyTuple_Check (), PyList_Check (), etc. 


There is also a high-level API to Python objects which is provided by the so-called ‘abstract’ interface — read 
Include/abstract.h for further details. It allows interfacing with any kind of Python sequence using calls 
like PySequence_Length(), PySequence_GetItem(), etc. as well as many other useful protocols such 
as numbers (PyNumber_ Index () et al.) and mappings in the PyMapping APIs. 


5.7 How do | use Py BuildValue() to create a tuple of arbitrary 
length? 


You can't. Use PyTuple_Pack () instead. 


5.8 How do | call an object's method from C? 


The PyObject_CallMethod () function can be used to call an arbitrary method of an object. The parameters 
are the object, the name of the method to call, a format string like that used with Py_BuildValue (), and the 
argument values: 


PyObject * 
PyObject_CallMethod (PyObject *object, const char *method_name, 
const char *arg_format, ...); 


This works for any object that has methods — whether built-in or user-defined. You are responsible for eventually 
Py_DECREF () ‘ing the return value. 


To call, e.g., a file object's “seek” method with arguments 10, 0 (assuming the file object pointer is “f”): 


res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0); 
if (res == NULL) ( 
. an exception occurred ... 


Py_DECREF (res); 


Note that since PyObject_Call0bject () always wants a tuple for the argument list, to call a function without 
arguments, pass “()” for the format, and to call a function with one argument, surround the argument in parentheses, 


e.g. “(i)”. 
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5.9 How do | catch the output from PyErr_Print() (or anything that 
prints to stdout/stderr)? 


In Python code, define an object that supports the write () method. Assign this object to sys.stdout and 
sys.stderr. Call print_error, or just allow the standard traceback mechanism to work. Then, the output will go 
wherever your write () method sends it. 


The easiest way to do this is to use the io. Stringlo class: 


>>> import io, sys 

>>> sys.stdout = io.StringI0O() 

>>> print ('foo') 

>>> print ('hello world!') 

>>> sys.stderr.write(sys.stdout.getvalue() ) 
foo 

hello world! 


A custom object to do the same would look like this: 


>>> import io, sys 
>>> class StdoutCatcher(io.TextIOBase): 
def __ init_ (self): 
self.data = [] 
def write(self, stuff): 
self.data.append (stuff) 


>>> import sys 

>>> sys.stdout = StdoutCatcher () 

>>> print ("foo") 

>>> print('hello world!') 

>>> sys.stderr.write(''.join(sys.stdout.data) ) 
foo 

hello world! 


5.10 How do I access a module written in Python from C? 


You can get a pointer to the module object as follows: 


module = PyImport_ImportModule("<modulename>") ; 


If the module hasn’t been imported yet (i.e. it is not yet present in sys.modules), this initializes the module; 
otherwise it simply returns the value of sys.modules["<modulename>"]. Note that it doesn’t enter the 
module into any namespace — it only ensures it has been initialized and is stored in sys .modules. 


You can then access the module’s attributes (i.e. any name defined in the module) as follows: 


attr = PyObject_GetAttrString(module, "<attrname>"); 


Calling PyObject_SetAttrString () to assign to variables in the module also works. 
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5.11 How do | interface to C++ objects from Python? 


Depending on your requirements, there are many approaches. To do this manually, begin by reading the “Extending 
and Embedding” document. Realize that for the Python run-time system, there isn't a whole lot of difference between 
C and C++ — so the strategy of building a new Python type around a C structure (pointer) type will also work for 
C++ objects. 


For C++ libraries, see Writing C is hard; are there any alternatives?. 


5.12 l added a module using the Setup file and the make fails; why? 


Setup must end in a newline, if there is no newline there, the build process fails. (Fixing this requires some ugly shell 
script hackery, and this bug is so minor that it doesn’t seem worth the effort.) 


5.13 How do | debug an extension? 


When using GDB with dynamically loaded extensions, you can’t set a breakpoint in your extension until your extension 
is loaded. 


In your . gdbinit file (or interactively), add the command: 


br _PyImport_LoadDynamicModule 


Then, when you run GDB: 


$ gdb /local/bin/python 
gdb) run myscript.py 


gdb) continue # repeat until your extension is loaded 
gdb) finish # so that your extension is loaded 

gdb) br myfunction.c:50 

gdb) continue 


5.14 | want to compile a Python module on my Linux system, but 
some files are missing. Why? 


Most packaged versions of Python don’t include the /usr/lib/python2.x/config/ directory, which con- 
tains various files required for compiling Python extensions. 


For Red Hat, install the python-devel RPM to get the necessary files. 


For Debian, run apt-get install python-dev. 


5.15 How do | tell “incomplete input” from “invalid input”? 


Sometimes you want to emulate the Python interactive interpreter's behavior, where it gives you a continuation prompt 
when the input is incomplete (e.g. you typed the start of an “if” statement or you didn't close your parentheses or 
triple string quotes), but it gives you a syntax error message immediately when the input is invalid. 


In Python you can use the codeop module, which approximates the parser’s behavior sufficiently. IDLE uses this, 
for example. 


The easiest way to do it in C is to call PyRun_InteractiveLoop () (perhaps in a separate thread) and let 
the Python interpreter handle the input for you. You can also set the PyOS_ReadlineFunctionPointer () 
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to point at your custom input function. See Modules/readline.c and Parser/myreadline.c for more 
hints. 


5.16 How do | find undefined g++ symbols _ builtin_new or 
_ pure_virtual? 


To dynamically load g++ extension modules, you must recompile Python, relink it using g++ (change LINKCC in the 
Python Modules Makefile), and link your extension module using g++ (e.g., g++ -shared -o mymodule.so 
mymodule. o). 


5.17 Can I create an object class with some methods implemented 
in C and others in Python (e.g. through inheritance)? 


Yes, you can inherit from built-in classes such as int, list, dict, etc. 


The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html) provides a way of doing this 
from C++ (1.e. you can inherit from an extension class written in C++ using the BPL). 
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SIX 


PYTHON ON WINDOWS FAQ 


6.1 How do I run a Python program under Windows? 


This is not necessarily a straightforward question. If you are already familiar with running programs from the Win- 
dows command line then everything will seem obvious; otherwise, you might need a little more guidance. 


Unless you use some sort of integrated development environment, you will end up typing Windows commands into 
what is referred to as a “Command prompt window”. Usually you can create such a window from your search bar 
by searching for cmd. You should be able to recognize when you have started such a window because you will see a 
Windows “command prompt”, which usually looks like this: 


Cys 


The letter may be different, and there might be other things after it, so you might just as easily see something like: 


D:\YourName\Projects\Python> 


depending on how your computer has been set up and what else you have recently done with it. Once you have started 
such a window, you are well on the way to running Python programs. 


You need to realize that your Python scripts have to be processed by another program called the Python interpreter. 
The interpreter reads your script, compiles it into bytecodes, and then executes the bytecodes to run your program. 
So, how do you arrange for the interpreter to handle your Python? 


First, you need to make sure that your command window recognises the word “py” as an instruction to start the 
interpreter. If you have opened a command window, you should try entering the command py and hitting return: 


C:\Users\YourName> py 


You should then see something like: 


Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel) Ju 
son win32 

Type "help", "copyright", "credits" or "license" for more information. 

>>> 


You have started the interpreter in “interactive mode”. That means you can enter Python statements or expressions 
interactively and have them executed or evaluated while you wait. This is one of Python’s strongest features. Check 
it by entering a few expressions of your choice and seeing the results: 


>>> print ("Hello") 


Hello 
>>> "Hello" * 3 
"HelloHelloHello' 


Many people use the interactive mode as a convenient yet highly programmable calculator. When you want to end 
your interactive Python session, call the exit () function or hold the Ct r1 key down while you enter a Z, then hit 
the “Enter” key to get back to your Windows command prompt. 
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You may also find that you have a Start-menu entry such as Start » Programs » Python 3.x » Python (command line) that 
results in you seeing the >>> prompt in a new window. If so, the window will disappear after you call the exit () 
function or enter the Ct r1—Z character; Windows is running a single “python” command in the window, and closes 
it when you terminate the interpreter. 


Now that we know the py command is recognized, you can give your Python script to it. You'll have to give either an 
absolute or a relative path to the Python script. Let’s say your Python script is located in your desktop and is named 
hello.py, and your command prompt is nicely opened in your home directory so you're seeing something similar 
to: 


C:\Users\YourName> 


So now you'll ask the py command to give your script to Python by typing py followed by your script path: 


C:\Users\YourName> py Desktop\hello.py 
hello 


6.2 How do | make Python scripts executable? 


On Windows, the standard Python installer already associates the .py extension with a file type (Python.File) and 
gives that file type an open command that runs the interpreter (D: \Program Files\Python\python.exe 
"%1" %*). This is enough to make scripts executable from the command prompt as ‘foo.py’. If you'd rather be 
able to execute the script by simple typing ‘foo’ with no extension you need to add .py to the PATHEXT environment 
variable. 


6.3 Why does Python sometimes take so long to start? 


Usually Python starts very quickly on Windows, but occasionally there are bug reports that Python suddenly begins 
to take a long time to start up. This is made even more puzzling because Python will work fine on other Windows 
systems which appear to be configured identically. 


The problem may be caused by a misconfiguration of virus checking software on the problem machine. Some virus 
scanners have been known to introduce startup overhead of two orders of magnitude when the scanner is configured 
to monitor all reads from the filesystem. Try checking the configuration of virus scanning software on your systems 
to ensure that they are indeed configured identically. McAfee, when configured to scan all file system read activity, 
is a particular offender. 


6.4 How do | make an executable from a Python script? 


See How can I create a stand-alone binary from a Python script? for a list of tools that can be used to make executables. 


6.5 Is a * . pyd file the same as a DLL? 


Yes, .pyd files are dll’s, but there are a few differences. If you have a DLL named foo. pyd, then it must have a 
function PyInit_foo(). You can then write Python “import foo”, and Python will search for foo.pyd (as well as 
foo.py, foo.pyc) and if it finds it, will attempt to call PyInit_foo () to initialize it. You do not link your .exe with 
foo.lib, as that would cause Windows to require the DLL to be present. 


Note that the search path for foo.pyd is PYTHONPATH, not the same as the path that Windows uses to search for 
foo.dll. Also, foo.pyd need not be present to run your program, whereas if you linked your program with a dll, the 
dll is required. Of course, foo.pyd is required if you want to say import foo. Ina DLL, linkage is declared in the 
source code with __ dec lspec (dllexport). In a .pyd, linkage is defined in a list of available functions. 
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6.6 How can | embed Python into a Windows application? 


Embedding the Python interpreter in a Windows app can be summarized as follows: 


1. Do not build Python into your .exe file directly. On Windows, Python must be a DLL to handle importing 
modules that are themselves DLL’s. (This is the first key undocumented fact.) Instead, link to pythonNN. 
a11; it is typically installed in C: \Windows\System. NN is the Python version, a number such as “33” 
for Python 3.3. 


You can link to Python in two different ways. Load-time linking means linking against pythonNN.1ib, 
while run-time linking means linking against pythonNN.d11l. (General note: pythonNN.lib is the 
so-called “import lib” corresponding to pythonNN.d11. It merely defines symbols for the linker.) 


Run-time linking greatly simplifies link options; everything happens at run time. Your code must load 
pythonNN.d11 using the Windows LoadLibraryEx () routine. The code must also use access rou- 
tines and data in pythonNN.d11 (that is, Pythons C API’s) using pointers obtained by the Windows 
GetProcAddress () routine. Macros can make using these pointers transparent to any C code that calls 
routines in Python’s C API. 


2. If you use SWIG, it is easy to create a Python “extension module” that will make the app’s data and methods 
available to Python. SWIG will handle just about all the grungy details for you. The result is C code that you 
link into your .exe file (!) You do not have to create a DLL file, and this also simplifies linking. 


3. SWIG will create an init function (a C function) whose name depends on the name of the extension module. 
For example, if the name of the module is leo, the init function will be called initleoQ). If you use SWIG 
shadow classes, as you should, the init function will be called initleoc(). This initializes a mostly hidden helper 
class used by the shadow class. 


The reason you can link the C code in step 2 into your .exe file is that calling the initialization function is 
equivalent to importing the module into Python! (This is the second key undocumented fact.) 


4. In short, you can use the following code to initialize the Python interpreter with your extension module. 


#include <Python.h> 


Py_Initialize(); // Initialize Python. 
initmyAppc(); // Initialize (import) the helper class. 
PyRun_SimpleString("import myApp"); // Import the shadow class. 


5. There are two problems with Python’s C API which will become apparent if you use a compiler other than 
MSVC, the compiler used to build pythonNN.dll. 


Problem 1: The so-called “Very High Level” functions that take FILE * arguments will not work in a multi- 
compiler environment because each compiler’s notion of a struct FILE will be different. From an imple- 
mentation standpoint these are very low level functions. 


Problem 2: SWIG generates the following code when generating wrappers to void functions: 


Py_INCREF (Py_None) ; 
_resultobj = Py_None; 
return _resultobj; 


Alas, Py_None is a macro that expands to a reference to a complex data structure called _Py_NoneStruct inside 
pythonNN.dll. Again, this code will fail in a mult-compiler environment. Replace such code by: 


return Py_BuildValue(""); 


It may be possible to use SWIG’s $t ypemap command to make the change automatically, though I have not 
been able to get this to work (Pm a complete SWIG newbie). 


6. Using a Python shell script to put up a Python interpreter window from inside your Windows app is not a 
good idea; the resulting window will be independent of your app’s windowing system. Rather, you (or the 
wxPython Window class) should create a “native” interpreter window. It is easy to connect that window to the 
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Python interpreter. You can redirect Python’s i/o to _any_ object that supports read and write, so all you need 
1s a Python object (defined in your extension module) that contains read() and write() methods. 


6.7 How do | keep editors from inserting tabs into my Python 
source? 


The FAQ does not recommend using tabs, and the Python style guide, PEP 8, recommends 4 spaces for distributed 
Python code; this is also the Emacs python-mode default. 


Under any editor, mixing tabs and spaces is a bad idea. MSVC is no different in this respect, and is easily configured 
to use spaces: Take Tools » Options » Tabs, and for file type “Default” set “Tab size” and “Indent size” to 4, and select 
the “Insert spaces” radio button. 


Python raises IndentationError or TabError if mixed tabs and spaces are causing problems in leading 
whitespace. You may also run the tabnanny module to check a directory tree in batch mode. 


6.8 How do | check for a keypress without blocking? 


Use the msvcrt module. This is a standard Windows-specific extension module. It defines a function kbhit () 
which checks whether a keyboard hit is present, and get ch () which gets one character without echoing it. 


6.9 How do | solve the missing api-ms-win-crt-runtime-11-1-0.dll er- 
ror? 


This can occur on Python 3.5 and later when using Windows 8.1 or earlier without all updates having been installed. 
First ensure your operating system is supported and is up to date, and if that does not resolve the issue, visit the 
Microsoft support page for guidance on manually installing the C Runtime update. 
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GRAPHIC USER INTERFACE FAQ 


7.1 General GUI Questions 


7.2 What GUI toolkits exist for Python? 


Standard builds of Python include an object-oriented interface to the Tcl/Tk widget set, called tkinter. This is probably 
the easiest to install (since it comes included with most binary distributions of Python) and use. For more info about 
Tk, including pointers to the source, see the Tcl/Tk home page. Tcl/Tk is fully portable to the macOS, Windows, 
and Unix platforms. 


Depending on what platform(s) you are aiming at, there are also several alternatives. A list of cross-platform and 
platform-specific GUI frameworks can be found on the python wiki. 


7.3 Tkinter questions 


7.3.1 How do I freeze Tkinter applications? 


Freeze is a tool to create stand-alone applications. When freezing Tkinter applications, the applications will not be 
truly stand-alone, as the application will still need the Tcl and Tk libraries. 


One solution is to ship the application with the Tcl and Tk libraries, and point to them at run-time using the 
TCL_LIBRARY and TK_LIBRARY environment variables. 


To get truly stand-alone applications, the Tcl scripts that form the library have to be integrated into the application 
as well. One tool supporting that is SAM (stand-alone modules), which is part of the Tix distribution (https://tix. 
sourceforge.net/). 


Build Tix with SAM enabled, perform the appropriate call to Tclsam_init (), etc. inside Python’s Modules/ 
tkappinit.c, and link with libtclsam and libtksam (you might include the Tix libraries as well). 


7.3.2 Can | have Tk events handled while waiting for 1/0? 


On platforms other than Windows, yes, and you don't even need threads! But you'll have to restructure your I/O code 
a bit. Tk has the equivalent of Xt's XtAddInput () call, which allows you to register a callback function which 
will be called from the Tk mainloop when I/O is possible on a file descriptor. See tkinter-file-handlers. 
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7.3.3 | can't get key bindings to work in Tkinter: why? 


An often-heard complaint is that event handlers bound to events with the bind () method don't get handled even 
when the appropriate key is pressed. 


The most common cause is that the widget to which the binding applies doesn’t have “keyboard focus”. Check out 
the Tk documentation for the focus command. Usually a widget is given the keyboard focus by clicking in it (but not 
for labels; see the takefocus option). 
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CHAPTER 
EIGHT 


“WHY IS PYTHON INSTALLED ON MY COMPUTER?” FAQ 


8.1 What is Python? 


Python is a programming language. It’s used for many different applications. It’s used in some high schools and 
colleges as an introductory programming language because Python is easy to learn, but it’s also used by professional 
software developers at places such as Google, NASA, and Lucasfilm Ltd. 


If you wish to learn more about Python, start with the Beginner's Guide to Python. 


8.2 Why is Python installed on my machine? 


If you find Python installed on your system but don't remember installing it, there are several possible ways it could 
have gotten there. 


e Perhaps another user on the computer wanted to learn programming and installed it; you'll have to figure out 
who's been using the machine and might have installed it. 


A third-party application installed on the machine might have been written in Python and included a Python 
installation. There are many such applications, from GUI programs to network servers and administrative 
scripts. 


Some Windows machines also have Python installed. At this writing we're aware of computers from Hewlett- 
Packard and Compaq that include Python. Apparently some of HP/Compaq’s administrative tools are written 
in Python. 


Many Unix-compatible operating systems, such as macOS and some Linux distributions, have Python installed 
by default; it’s included in the base installation. 


8.3 Can | delete Python? 


That depends on where Python came from. 


If someone installed it deliberately, you can remove it without hurting anything. On Windows, use the Add/Remove 
Programs icon in the Control Panel. 


If Python was installed by a third-party application, you can also remove it, but that application will no longer work. 
You should use that application’s uninstaller rather than removing Python directly. 


If Python came with your operating system, removing it is not recommended. If you remove it, whatever tools were 
written in Python will no longer run, and some of them might be important to you. Reinstalling the whole system 
would then be required to fix things again. 


73 


Python Frequently Asked Questions, Release 3.11.1 


74 


Chapter 8. “Why is Python Installed on my Computer?” FAQ 


APPENDIX 
A 


GLOSSARY 


>>> The default Python prompt of the interactive shell. Often seen for code examples which can be executed 
interactively in the interpreter. 


. Can refer to: 


e The default Python prompt of the interactive shell when entering the code for an indented code block, 
when within a pair of matching left and right delimiters (parentheses, square brackets, curly braces or 
triple quotes), or after specifying a decorator. 


e The Ellipsis built-in constant. 


2to3 A tool that tries to convert Python 2.x code to Python 3.x code by handling most of the incompatibilities which 
can be detected by parsing the source and traversing the parse tree. 


2to3 is available in the standard library as 1ib2to3; a standalone entry point is provided as Tools/ 
scripts/2to3. See 2to3-reference. 


abstract base class Abstract base classes complement duck-typing by providing a way to define interfaces when 
other techniques like hasattr () would be clumsy or subtly wrong (for example with magic methods). 
ABCs introduce virtual subclasses, which are classes that don't inherit from a class but are still recognized 
by isinstance() and issubclass (); see the abc module documentation. Python comes with many 
built-in ABCs for data structures (in the collections. abc module), numbers (in the numbers module), 
streams (in the io module), import finders and loaders (in the importlib.abc module). You can create 
your own ABCs with the abc module. 


annotation A label associated with a variable, a class attribute or a function parameter or return value, used by 
convention as a type hint. 


Annotations of local variables cannot be accessed at runtime, but annotations of global variables, class at- 
tributes, and functions are stored in the __annotations__ special attribute of modules, classes, and func- 
tions, respectively. 


See variable annotation, function annotation, PEP 484 and PEP 526, which describe this functionality. Also 
see annotations-howto for best practices on working with annotations. 


argument A value passed to a function (or method) when calling the function. There are two kinds of argument: 


e keyword argument: an argument preceded by an identifier (e.g. name=) in a function call or passed as a 
value in a dictionary preceded by * *. For example, 3 and 5 are both keyword arguments in the following 
calls to complex (): 


complex (real=3, imag=5) 
complex(**{'real': 3, 'imag': 5}) 


e positional argument: an argument that is not a keyword argument. Positional arguments can appear at the 
beginning of an argument list and/or be passed as elements of an iferable preceded by *. For example, 3 
and 5 are both positional arguments in the following calls: 


complex(3, 5) 
complex (* (3, 5)) 
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Arguments are assigned to the named local variables in a function body. See the calls section for the rules 
governing this assignment. Syntactically, any expression can be used to represent an argument; the evaluated 
value is assigned to the local variable. 


See also the parameter glossary entry, the FAQ question on the difference between arguments and parameters, 
and PEP 362. 


asynchronous context manager An object which controls the environment seen in an async with statement by 
defining__aenter__ () and__aexit__ () methods. Introduced by PEP 492. 


asynchronous generator A function which returns an asynchronous generator iterator. It looks like a coroutine 
function defined with async def except that it contains yield expressions for producing a series of values 
usable in an async for loop. 


Usually refers to an asynchronous generator function, but may refer to an asynchronous generator iterator in 
some contexts. In cases where the intended meaning isn’t clear, using the full terms avoids ambiguity. 


An asynchronous generator function may contain await expressions as well as async for, and async 
with statements. 


asynchronous generator iterator An object created by a asynchronous generator function. 


This is an asynchronous iterator which when called using the __anext__() method returns an awaitable 
object which will execute the body of the asynchronous generator function until the next yield expression. 


Each yield temporarily suspends processing, remembering the location execution state (including local vari- 
ables and pending try-statements). When the asynchronous generator iterator effectively resumes with another 
awaitable returned by__ ane xt___(), it picks up where it left off. See PEP 492 and PEP 525. 


asynchronous iterable An object, that can be used in an async for statement. Must return an asynchronous 
iterator from its__aiter__ () method. Introduced by PEP 492. 


asynchronous iterator An object that implements the __aiter__() and __anext__() methods. 
__anext__ must return an awaitable object. async for resolves the awaitables returned by an 
asynchronous iterators __anext__() method until it raises a StopAsyncIteration exception. 
Introduced by PEP 492. 


attribute A value associated with an object which is usually referenced by name using dotted expressions. For 
example, if an object o has an attribute a it would be referenced as o.a. 


It is possible to give an object an attribute whose name is not an identifier as defined by identifiers, for example 
using setattr (), if the object allows it. Such an attribute will not be accessible using a dotted expression, 
and would instead need to be retrieved with getattr (). 


awaitable An object that can be used in an await expression. Can be a coroutine or an object with an 
__await__ () method. See also PEP 492. 


BDFL Benevolent Dictator For Life, a.k.a. Guido van Rossum, Python's creator. 


binary file A file object able to read and write bytes-like objects. Examples of binary files are files opened in binary 
mode ('rb', 'wb' or 'rb+'), sys.stdin.buffer, sys.stdout .buffer, and instances of io. 
BytesI0 and gzip.GzipFile. 


See also text file for a file object able to read and write st r objects. 


borrowed reference In Python’s C API, a borrowed reference is a reference to an object. It does not modify the 
object reference count. It becomes a dangling pointer if the object is destroyed. For example, a garbage 
collection can remove the last strong reference to the object and so destroy it. 


Calling Py_INCREF () on the borrowed reference is recommended to convert it to a strong reference in- 
place, except when the object cannot be destroyed before the last usage of the borrowed reference. The 
Py_NewRef () function can be used to create a new strong reference. 


bytes-like object An object that supports the bufferobjects and can export a C-contiguous buffer. This includes all 
bytes, bytearray, and array. array objects, as well as many common memoryview objects. Bytes- 
like objects can be used for various operations that work with binary data; these include compression, saving 
to a binary file, and sending over a socket. 
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Some operations need the binary data to be mutable. The documentation often refers to these as “read- 
write bytes-like objects”. Example mutable buffer objects include bytearray and a memoryview of a 
bytearray. Other operations require the binary data to be stored in immutable objects (“read-only bytes- 
like objects”); examples of these include bytes and a memoryview of a bytes object. 


bytecode Python source code is compiled into bytecode, the internal representation of a Python program in the 
CPython interpreter. The bytecode is also cached in . pyc files so that executing the same file is faster the 
second time (recompilation from source to bytecode can be avoided). This “intermediate language” is said 
to run on a virtual machine that executes the machine code corresponding to each bytecode. Do note that 
bytecodes are not expected to work between different Python virtual machines, nor to be stable between Python 
releases. 


A list of bytecode instructions can be found in the documentation for the dis module. 


callable A callable is an object that can be called, possibly with a set of arguments (see argument), with the following 
syntax: 


callable (argument1, argument2, ...) 


A function, and by extension a method, is a callable. An instance of a class that implements the__ca11__ () 
method is also a callable. 


callback A subroutine function which is passed as an argument to be executed at some point in the future. 


class A template for creating user-defined objects. Class definitions normally contain method definitions which 
operate on instances of the class. 


class variable A variable defined in a class and intended to be modified only at class level (i.e., not in an instance of 
the class). 


complex number An extension of the familiar real number system in which all numbers are expressed as a sum of 
a real part and an imaginary part. Imaginary numbers are real multiples of the imaginary unit (the square root 
of —1), often written i in mathematics or j in engineering. Python has built-in support for complex numbers, 
which are written with this latter notation; the imaginary part is written with a j suffix, e.g., 3+13. To get 
access to complex equivalents of the math module, use cmath. Use of complex numbers is a fairly advanced 
mathematical feature. If you’re not aware of a need for them, it’s almost certain you can safely ignore them. 


context manager An object which controls the environment seen in a with statement by defining __ ent er___() 
and__exit__ () methods. See PEP 343. 


context variable A variable which can have different values depending on its context. This is similar to Thread- 
Local Storage in which each execution thread may have a different value for a variable. However, with context 
variables, there may be several contexts in one execution thread and the main usage for context variables is to 
keep track of variables in concurrent asynchronous tasks. See contextva:rs. 


contiguous A buffer is considered contiguous exactly if it is either C-contiguous or Fortran contiguous. Zero- 
dimensional buffers are C and Fortran contiguous. In one-dimensional arrays, the items must be laid out in 
memory next to each other, in order of increasing indexes starting from zero. In multidimensional C-contiguous 
arrays, the last index varies the fastest when visiting items in order of memory address. However, in Fortran 
contiguous arrays, the first index varies the fastest. 


coroutine Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited 
at another point. Coroutines can be entered, exited, and resumed at many different points. They can be 
implemented with the async def statement. See also PEP 492. 


coroutine function A function which returns a coroutine object. A coroutine function may be defined with the 
async def statement, and may contain await, async for, and async with keywords. These were 
introduced by PEP 492. 


CPython The canonical implementation of the Python programming language, as distributed on python.org. The 
term “CPython” is used when necessary to distinguish this implementation from others such as Jython or 
IronPython. 


decorator A function returning another function, usually applied as a function transformation using the (wrapper 
syntax. Common examples for decorators are classmethod () and staticmethod (). 
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The decorator syntax is merely syntactic sugar, the following two function definitions are semantically equiv- 
alent: 


def f(arg): 
f = staticmethod(f) 


@staticmethod 
def f(arg): 


The same concept exists for classes, but is less commonly used there. See the documentation for function 
definitions and class definitions for more about decorators. 


descriptor Any object which defines the methods __get__ (),__set__ (),or__delete__ (). When a class 
attribute is a descriptor, its special binding behavior is triggered upon attribute lookup. Normally, using a.b to 
get, set or delete an attribute looks up the object named b in the class dictionary for a, but if bis a descriptor, 
the respective descriptor method gets called. Understanding descriptors is a key to a deep understanding of 
Python because they are the basis for many features including functions, methods, properties, class methods, 
static methods, and reference to super classes. 


For more information about descriptors’ methods, see descriptors or the Descriptor How To Guide. 


dictionary An associative array, where arbitrary keys are mapped to values. The keys can be any object with 
__hash__() and___eq___() methods. Called a hash in Perl. 


dictionary comprehension A compact way to process all or part of the elements in an iterable and return a dic- 
tionary with the results. results = {n: n ** 2 for n in range(10) } generates a dictionary 
containing key n mapped to value n ** 2. See comprehensions. 


dictionary view The objects returned from dict .keys (), dict.values(),anddict.items () are called 
dictionary views. They provide a dynamic view on the dictionary’s entries, which means that when the dic- 
tionary changes, the view reflects these changes. To force the dictionary view to become a full list use 
list (dictview). See dict-views. 


docstring A string literal which appears as the first expression in a class, function or module. While ignored when 
the suite is executed, it is recognized by the compiler and put into the __doc___ attribute of the enclosing 
class, function or module. Since it is available via introspection, it is the canonical place for documentation of 
the object. 


duck-typing A programming style which does not look at an object’s type to determine if it has the right interface; 
instead, the method or attribute is simply called or used (“If it looks like a duck and quacks like a duck, it must 
be a duck.”) By emphasizing interfaces rather than specific types, well-designed code improves its flexibility 
by allowing polymorphic substitution. Duck-typing avoids tests using type () or isinstance (). (Note, 
however, that duck-typing can be complemented with abstract base classes.) Instead, it typically employs 
hasattr () tests or EAFP programming. 


EAFP Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of 
valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is 
characterized by the presence of many try and except statements. The technique contrasts with the LBYL 
style common to many other languages such as C. 


expression A piece of syntax which can be evaluated to some value. In other words, an expression is an accumulation 
of expression elements like literals, names, attribute access, operators or function calls which all return a value. 
In contrast to many other languages, not all language constructs are expressions. There are also statements 
which cannot be used as expressions, such as while. Assignments are also statements, not expressions. 


extension module A module written in C or C++, using Python’s C API to interact with the core and with user code. 


f-string String literals prefixed with ' £ ' or 'F ' are commonly called “f-strings” which is short for formatted string 
literals. See also PEP 498. 


file object An object exposing a file-oriented API (with methods such as read () or write ()) to an underlying 
resource. Depending on the way it was created, a file object can mediate access to a real on-disk file or to another 
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type of storage or communication device (for example standard input/output, in-memory buffers, sockets, 
pipes, etc.). File objects are also called file-like objects or streams. 


There are actually three categories of file objects: raw binary files, buffered binary files and text files. Their 
interfaces are defined in the io module. The canonical way to create a file object is by using the open () 
function. 


file-like object A synonym for file object. 


filesystem encoding and error handler Encoding and error handler used by Python to decode bytes from the op- 
erating system and encode Unicode to the operating system. 


The filesystem encoding must guarantee to successfully decode all bytes below 128. If the file system encoding 
fails to provide this guarantee, API functions can raise UnicodeError. 


The sys.getfilesystemencoding() and sys.getfilesystemencodeerrors () functions 
can be used to get the filesystem encoding and error handler. 


The filesystem encoding and error handler are configured at Python startup by the PyConfig_Read () func- 
tion: see filesystem_encoding and filesystem_errors members of PyConfig. 


See also the locale encoding. 
finder An object that tries to find the loader for a module that is being imported. 


Since Python 3.3, there are two types of finder: meta path finders for use with sys .meta_path, and path 
entry finders for use with sys .path_hooks. 


See PEP 302, PEP 420 and PEP 451 for much more detail. 


floor division Mathematical division that rounds down to nearest integer. The floor division operator is //. For 
example, the expression 11 // 4 evaluates to 2 in contrast to the 2 . 75 returned by float true division. Note 
that (-11) // 4is -3 because that is -2 . 75 rounded downward. See PEP 238. 


function A series of statements which returns some value to a caller. It can also be passed zero or more arguments 
which may be used in the execution of the body. See also parameter, method, and the function section. 


function annotation An annotation of a function parameter or return value. 


Function annotations are usually used for type hints: for example, this function is expected to take two int 
arguments and is also expected to have an int return value: 


def sum_two_numbers(a: int, b: int) -> int: 
return a + b 


Function annotation syntax is explained in section function. 


See variable annotation and PEP 484, which describe this functionality. Also see annotations-howto for best 
practices on working with annotations. 


_future__ A future statement, from __future__ import <feature>, directs the compiler to compile 
the current module using syntax or semantics that will become standard in a future release of Python. The 
__future__ module documents the possible values of feature. By importing this module and evaluating its 
variables, you can see when a new feature was first added to the language and when it will (or did) become the 


default: 
>>> import _ future __ 
>>> _ future__.division 


_Feature((2, 2, 0, "alpha", 2), (3, 0, 0, 'alpha', 0), 8192) 


garbage collection The process of freeing memory when it is not used anymore. Python performs garbage collection 
via reference counting and a cyclic garbage collector that is able to detect and break reference cycles. The 
garbage collector can be controlled using the gc module. 


generator A function which returns a generator iterator. It looks like a normal function except that it contains yield 
expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the 
next () function. 
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Usually refers to a generator function, but may refer to a generator iterator in some contexts. In cases where 
the intended meaning isn’t clear, using the full terms avoids ambiguity. 


generator iterator An object created by a generator function. 


Each yield temporarily suspends processing, remembering the location execution state (including local vari- 
ables and pending try-statements). When the generator iterator resumes, it picks up where it left off (in contrast 
to functions which start fresh on every invocation). 


generator expression An expression that returns an iterator. It looks like a normal expression followed by a for 
clause defining a loop variable, range, and an optional if clause. The combined expression generates values 
for an enclosing function: 


>>> sum(i*i for i in range(10)) # sum of squares 0, 1, 4, ... 81 
285 


generic function A function composed of multiple functions implementing the same operation for different types. 
Which implementation should be used during a call is determined by the dispatch algorithm. 


See also the single dispatch glossary entry, the functools.singledispatch() decorator, and PEP 
443. 


generic type A type that can be parameterized; typically a container class such as list or dict. Used for type 
hints and annotations. 


For more details, see generic alias types, PEP 483, PEP 484, PEP 585, and the typing module. 
GIL See global interpreter lock. 


global interpreter lock The mechanism used by the CPython interpreter to assure that only one thread executes 
Python bytecode at a time. This simplifies the CPython implementation by making the object model (including 
critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter 
makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by 
multi-processor machines. 


However, some extension modules, either standard or third-party, are designed so as to release the GIL when 
doing computationally intensive tasks such as compression or hashing. Also, the GIL is always released when 
doing I/O. 


Past efforts to create a “free-threaded” interpreter (one which locks shared data at a much finer granularity) 
have not been successful because performance suffered in the common single-processor case. It is believed 
that overcoming this performance issue would make the implementation much more complicated and therefore 
costlier to maintain. 


hash-based pyc A bytecode cache file that uses the hash rather than the last-modified time of the corresponding 
source file to determine its validity. See pyc-invalidation. 


hashable An object is hashable if it has a hash value which never changes during its lifetime (it needs a 
__hash__() method), and can be compared to other objects (it needs an ___eq__ () method). Hashable 
objects which compare equal must have the same hash value. 


Hashability makes an object usable as a dictionary key and a set member, because these data structures use the 
hash value internally. 


Most of Python’s immutable built-in objects are hashable; mutable containers (such as lists or dictionaries) 
are not; immutable containers (such as tuples and frozensets) are only hashable if their elements are hashable. 
Objects which are instances of user-defined classes are hashable by default. They all compare unequal (except 
with themselves), and their hash value is derived from their id (). 


IDLE An Integrated Development and Learning Environment for Python. idle is a basic editor and interpreter 
environment which ships with the standard distribution of Python. 


immutable An object with a fixed value. Immutable objects include numbers, strings and tuples. Such an object 
cannot be altered. A new object has to be created if a different value has to be stored. They play an important 
role in places where a constant hash value is needed, for example as a key in a dictionary. 
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import path A list of locations (or path entries) that are searched by the path based finder for modules to import. 
During import, this list of locations usually comes from sys.path, but for subpackages it may also come 
from the parent package's__path__ attribute. 


importing The process by which Python code in one module is made available to Python code in another module. 
importer An object that both finds and loads a module; both a finder and loader object. 


interactive Python has an interactive interpreter which means you can enter statements and expressions at the in- 
terpreter prompt, immediately execute them and see their results. Just launch python with no arguments 
(possibly by selecting it from your computer's main menu). It is a very powerful way to test out new ideas or 
inspect modules and packages (remember help (x) ). 


interpreted Python is an interpreted language, as opposed to a compiled one, though the distinction can be blurry be- 
cause of the presence of the bytecode compiler. This means that source files can be run directly without explic- 
itly creating an executable which is then run. Interpreted languages typically have a shorter development/debug 
cycle than compiled ones, though their programs generally also run more slowly. See also interactive. 


interpreter shutdown When asked to shut down, the Python interpreter enters a special phase where it gradually 
releases all allocated resources, such as modules and various critical internal structures. It also makes several 
calls to the garbage collector. This can trigger the execution of code in user-defined destructors or weakref 
callbacks. Code executed during the shutdown phase can encounter various exceptions as the resources it 
relies on may not function anymore (common examples are library modules or the warnings machinery). 


The main reason for interpreter shutdown is that the __main__ module or the script being run has finished 
executing. 


iterable An object capable of returning its members one at a time. Examples of iterables include all sequence 
types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of 
any classes you define with an__iter___() method or witha __getitem__() method that implements 
sequence semantics. 


Iterables can be used in a for loop and in many other places where a sequence is needed (zip (), map (), 
...). When an iterable object is passed as an argument to the built-in function iter (), it returns an iterator 
for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not 
necessary to call iter () or deal with iterator objects yourself. The for statement does that automatically for 
you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also iterator, 
sequence, and generator. 


iterator An object representing a stream of data. Repeated calls to the iterator’s __next___() method (or passing 
it to the built-in function next () ) return successive items in the stream. When no more data are available 
a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any fur- 
ther calls to its __next__ () method just raise StopIteration again. Iterators are required to have an 
__¡iter__ () method that returns the iterator object itself so every iterator is also iterable and may be used 
in most places where other iterables are accepted. One notable exception is code which attempts multiple 
iteration passes. A container object (such as a 1ist) produces a fresh new iterator each time you pass it to the 
iter () function or use it ina for loop. Attempting this with an iterator will just return the same exhausted 
iterator object used in the previous iteration pass, making it appear like an empty container. 


More information can be found in typeiter. 


CPython implementation detail: CPython does not consistently apply the requirement that an iterator define 
EEE (i 


key function A key function or collation function is a callable that returns a value used for sorting or ordering. For 
example, locale.strxfrm() is used to produce a sort key that is aware of locale specific sort conventions. 


A number of tools in Python accept key functions to control how elements are ordered or grouped. They include 
min (), max(), sorted(), list.sort(), heapq.merge(), heapq.nsmallest (), heapq. 
nlargest (), and itertools.groupby(). 


There are several ways to create a key function. For example. the str . lower () method can serve as a key 
function for case insensitive sorts. Alternatively, a key function can be built from a 1 ambda expression such as 
lambda r: (r[0], r[2]). Also, operator.attrgetter (), operator.itemgetter (), 
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and operator .methodcaller () are three key function constructors. See the Sorting HOW TO for 
examples of how to create and use key functions. 


keyword argument See argument. 


lambda An anonymous inline function consisting of a single expression which is evaluated when the function is 
called. The syntax to create a lambda function is Lambda [parameters]: expression 


LBYL Look before you leap. This coding style explicitly tests for pre-conditions before making calls or lookups. 
This style contrasts with the EAFP approach and is characterized by the presence of many if statements. 


In a multi-threaded environment, the LBYL approach can risk introducing a race condition between “the 
looking” and “the leaping”. For example, the code, if key in mapping: return mapping[key] 
can fail if another thread removes key from mapping after the test, but before the lookup. This issue can be 
solved with locks or by using the EAFP approach. 


locale encoding On Unix, it is the encoding of the LC_CTYPE locale. It can be set with locale. 
setlocale(locale.LC_CTYPE, new_locale). 


On Windows, it is the ANSI code page (ex: "cp1252"). 

On Android and VxWorks, Python uses "ut £-8" as the locale encoding. 
locale.getencoding() can be used to get the locale encoding. 

See also the filesystem encoding and error handler. 


list A built-in Python sequence. Despite its name it is more akin to an array in other languages than to a linked list 
since access to elements is O(1). 


list comprehension A compact way to process all or part of the elements in a sequence and return a list with the re- 
sults. result = ['{:#04x}'.format(x) for x in range(256) if x % 2 == 0] gen- 
erates a list of strings containing even hex numbers (Ox..) in the range from 0 to 255. The i f clause is optional. 


If omitted, all elements in range (256) are processed. 


loader An object that loads a module. It must define a method named load_module(). A loader is typically 
returned by a finder. See PEP 302 for details and importlib.abc.Loader for an abstract base class. 


magic method An informal synonym for special method. 


mapping A container object that supports arbitrary key lookups and implements the methods specified 
in the collections.abc.Mapping or collections.abc.MutableMapping abstract base 
classes. Examples include dict, collections.defaultdict, collections.OrderedDict and 
collections.Counter. 


meta path finder A finder returned by a search of sys .meta_path. Meta path finders are related to, but different 
from path entry finders. 


See importlib.abc.MetaPathFinder for the methods that meta path finders implement. 


metaclass The class of a class. Class definitions create a class name, a class dictionary, and a list of base classes. 
The metaclass is responsible for taking those three arguments and creating the class. Most object oriented 
programming languages provide a default implementation. What makes Python special is that it is possible to 
create custom metaclasses. Most users never need this tool, but when the need arises, metaclasses can provide 
powerful, elegant solutions. They have been used for logging attribute access, adding thread-safety, tracking 
object creation, implementing singletons, and many other tasks. 


More information can be found in metaclasses. 


method A function which is defined inside a class body. If called as an attribute of an instance of that class, the 
method will get the instance object as its first argument (which is usually called se1 f). See function and nested 
scope. 


method resolution order Method Resolution Order is the order in which base classes are searched for a member 
during lookup. See The Python 2.3 Method Resolution Order for details of the algorithm used by the Python 
interpreter since the 2.3 release. 
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module An object that serves as an organizational unit of Python code. Modules have a namespace containing 
arbitrary Python objects. Modules are loaded into Python by the process of importing. 


See also package. 


module spec A namespace containing the import-related information used to load a module. An instance of 
importlib.machinery.ModuleSpec. 


MRO See method resolution order. 
mutable Mutable objects can change their value but keep their id (). See also immutable. 


named tuple The term “named tuple” applies to any type or class that inherits from tuple and whose indexable 
elements are also accessible using named attributes. The type or class may have other features as well. 


Several built-in types are named tuples, including the values returned by time. localtime() and os. 
stat (). Another example is sys. float_info: 


>>> sys.float_info[1] # indexed access 
1024 

>>> sys.float_info.max_exp # named field access 
1024 

>>> isinstance(sys.float_info, tuple) # kind of tuple 

True 


Some named tuples are built-in types (such as the above examples). Alternatively, a named tuple can be created 
from a regular class definition that inherits from tuple and that defines named fields. Such a class can be 
written by hand or it can be created with the factory function collections.namedtuple (). The latter 
technique also adds some extra methods that may not be found in hand-written or built-in named tuples. 


namespace The place where a variable is stored. Namespaces are implemented as dictionaries. There are the local, 
global and built-in namespaces as well as nested namespaces in objects (in methods). Namespaces support 
modularity by preventing naming conflicts. For instance, the functions builtins.open and os.open () 
are distinguished by their namespaces. Namespaces also aid readability and maintainability by making it clear 
which module implements a function. For instance, writing random. seed () oritertools.islice() 
makes it clear that those functions are implemented by the random and itertools modules, respectively. 


namespace package A PEP 420 package which serves only as a container for subpackages. Namespace packages 
may have no physical representation, and specifically are not like a regular package because they have no 
__ init__.py file. 


See also module. 


nested scope The ability to refer to a variable in an enclosing definition. For instance, a function defined inside 
another function can refer to variables in the outer function. Note that nested scopes by default work only for 
reference and not for assignment. Local variables both read and write in the innermost scope. Likewise, global 
variables read and write to the global namespace. The nonlocal allows writing to outer scopes. 


new-style class Old name for the flavor of classes now used for all class objects. In earlier Python versions, 
only new-style classes could use Python’s newer, versatile features like __slots__, descriptors, properties, 
__ getattribute__ (), class methods, and static methods. 


object Any data with state (attributes or value) and defined behavior (methods). Also the ultimate base class of any 
new-style class. 


package A Python module which can contain submodules or recursively, subpackages. Technically, a package is a 
Python module witha __ path_ attribute. 


See also regular package and namespace package. 


parameter A named entity in a function (or method) definition that specifies an argument (or in some cases, argu- 
ments) that the function can accept. There are five kinds of parameter: 


e positional-or-keyword: specifies an argument that can be passed either positionally or as a keyword argu- 
ment. This is the default kind of parameter, for example foo and bar in the following: 
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def func(foo, bar=None) : 


e positional-only: specifies an argument that can be supplied only by position. Positional-only parameters 
can be defined by including a / character in the parameter list of the function definition after them, for 
example posonly! and posonly2 in the following: 


def func(posonly1, posonly2, /, positional_or_keyword): 


e keyword-only: specifies an argument that can be supplied only by keyword. Keyword-only parameters can 
be defined by including a single var-positional parameter or bare * in the parameter list of the function 
definition before them, for example kw_only1 and kw_only2 in the following: 


def func(arg, *, kw_only1, kw_only2): 


e var-positional: specifies that an arbitrary sequence of positional arguments can be provided (in addition 
to any positional arguments already accepted by other parameters). Such a parameter can be defined by 
prepending the parameter name with *, for example args in the following: 


def func(*args, **kwargs): 


e var-keyword: specifies that arbitrarily many keyword arguments can be provided (in addition to any key- 
word arguments already accepted by other parameters). Such a parameter can be defined by prepending 
the parameter name with * *, for example kwargs in the example above. 


Parameters can specify both optional and required arguments, as well as default values for some optional 
arguments. 


See also the argument glossary entry, the FAQ question on the difference between arguments and parameters, 
the inspect .Parameter class, the function section, and PEP 362. 


path entry A single location on the import path which the path based finder consults to find modules for importing. 


path entry finder A finder returned by a callable on sys . path_hooks (i.e. a path entry hook) which knows how 
to locate modules given a path entry. 


See importlib.abc.PathEntryFinder for the methods that path entry finders implement. 


path entry hook A callable on the sys .path_hook list which returns a path entry finder if it knows how to find 
modules on a specific path entry. 


path based finder One of the default meta path finders which searches an import path for modules. 


path-like object An object representing a file system path. A path-like object is either a str or bytes object 
representing a path, or an object implementing the os . PathLike protocol. An object that supports the os . 
PathLike protocol can be converted to a str or bytes file system path by calling the os. fspath () 
function; os .fsdecode () andos.fsencode () can be used to guarantee a st r or bytes result instead, 
respectively. Introduced by PEP 519. 


PEP Python Enhancement Proposal. A PEP is a design document providing information to the Python community, 
or describing a new feature for Python or its processes or environment. PEPs should provide a concise technical 
specification and a rationale for proposed features. 


PEPs are intended to be the primary mechanisms for proposing major new features, for collecting community 
input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is 
responsible for building consensus within the community and documenting dissenting opinions. 


See PEP 1. 


portion A set of files in a single directory (possibly stored in a zip file) that contribute to a namespace package, as 
defined in PEP 420. 


positional argument See argument. 


provisional API A provisional API is one which has been deliberately excluded from the standard library’s back- 
wards compatibility guarantees. While major changes to such interfaces are not expected, as long as they are 
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marked provisional, backwards incompatible changes (up to and including removal of the interface) may occur 
if deemed necessary by core developers. Such changes will not be made gratuitously — they will occur only if 
serious fundamental flaws are uncovered that were missed prior to the inclusion of the API. 


Even for provisional APIs, backwards incompatible changes are seen as a “solution of last resort” - every 
attempt will still be made to find a backwards compatible resolution to any identified problems. 


This process allows the standard library to continue to evolve over time, without locking in problematic design 
errors for extended periods of time. See PEP 411 for more details. 


provisional package See provisional API. 


Python 3000 Nickname for the Python 3.x release line (coined long ago when the release of version 3 was something 
in the distant future.) This is also abbreviated “Py3k”. 


Pythonic An idea or piece of code which closely follows the most common idioms of the Python language, rather 
than implementing code using concepts common to other languages. For example, a common idiom in Python 
is to loop over all elements of an iterable using a for statement. Many other languages don’t have this type of 
construct, so people unfamiliar with Python sometimes use a numerical counter instead: 


for i in range(len(food)): 
print (food[i]) 


As opposed to the cleaner, Pythonic method: 


for piece in food: 
print (piece) 


qualified name A dotted name showing the “path” from a module’s global scope to a class, function or method 
defined in that module, as defined in PEP 3155. For top-level functions and classes, the qualified name is the 
same as the object’s name: 


>>> class C: 
class D: 
def meth(self): 
pass 
>>> C._ qualname__ 
Li eau 
>>> C.D.__qualname__ 
ED 
>>> C.D.meth._ qualname__ 
'C.D.meth' 


When used to refer to modules, the fully qualified name means the entire dotted path to the module, including 
any parent packages, e.g. email.mime.text: 


>>> import email.mime.text 
>>> email.mime.text. name __ 
'"email.mime.text' 


reference count The number of references to an object. When the reference count of an object drops to zero, it is 
deallocated. Reference counting is generally not visible to Python code, but it is a key element of the CPython 
implementation. Programmers can call the sys .getrefcount () function to return the reference count 
for a particular object. 


regular package A traditional package, such as a directory containing an__ init__.py file. 
See also namespace package. 


__slots__ A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminat- 
ing instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved 
for rare cases where there are large numbers of instances in a memory-critical application. 
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sequence An iferable which supports efficient element access using integer indices via the __getitem__ () spe- 
cial method and defines a __len__ () method that returns the length of the sequence. Some built-in se- 
quence types are list, str, tuple, and bytes. Note that dict also supports __getitem__ () and 
__len__ (), butis considered a mapping rather than a sequence because the lookups use arbitrary immutable 
keys rather than integers. 


The collections.abc. Sequence abstract base class defines a much richer interface that goes be- 
yond just __getitem__ () and len__ (), adding count (), index (), contains__ (), and 
__reversed__(). Types that implement this expanded interface can be registered explicitly using 
register (). 


set comprehension A compact way to process all or part of the elements in an iterable and return a set with the 
results. results = {c for c in 'abracadabra' if c not in 'abc') generates the set 
of strings ('r', 'd'}. See comprehensions. 


single dispatch A form of generic function dispatch where the implementation is chosen based on the type of a 
single argument. 


slice An object usually containing a portion of a sequence. A slice is created using the subscript notation, [] with 
colons between numbers when several are given, such as in variable_name[1:3:5]. The bracket (sub- 
script) notation uses slice objects internally. 


special method A method that is called implicitly by Python to execute a certain operation on a type, such as addition. 
Such methods have names starting and ending with double underscores. Special methods are documented in 
specialnames. 


statement A statement is part of a suite (a “block” of code). A statement is either an expression or one of several 
constructs with a keyword, such as if, while or for. 


strong reference In Python’s C API, a strong reference is a reference to an object which increments the object’s 
reference count when it is created and decrements the object’s reference count when it is deleted. 


The Py_NewRef() function can be used to create a strong reference to an object. Usually, the 
Py_DECREF () function must be called on the strong reference before exiting the scope of the strong refer- 
ence, to avoid leaking one reference. 


See also borrowed reference. 


text encoding A string in Python is a sequence of Unicode code points (in range U+0000-U+10FFFE). To store 
or transfer a string, it needs to be serialized as a sequence of bytes. 


Serializing a string into a sequence of bytes is known as “encoding”, and recreating the string from the sequence 
of bytes is known as “decoding”. 


There are a variety of different text serialization codecs, which are collectively referred to as “text encodings”. 


text file A file object able to read and write st r objects. Often, a text file actually accesses a byte-oriented datastream 
and handles the text encoding automatically. Examples of text files are files opened in text mode ('r' or 'w'), 
sys.stdin, sys.stdout, and instances of io. StringIO. 


See also binary file for a file object able to read and write bytes-like objects. 


triple-quoted string A string which is bound by three instances of either a quotation mark (”) or an apostrophe 
(©). While they don’t provide any functionality not available with single-quoted strings, they are useful for a 
number of reasons. They allow you to include unescaped single and double quotes within a string and they can 
span multiple lines without the use of the continuation character, making them especially useful when writing 
docstrings. 


type The type of a Python object determines what kind of object it is; every object has a type. An object’s type is 
accessible as its __ class__ attribute or can be retrieved with type (obj). 


type alias A synonym for a type, created by assigning the type to an identifier. 


Type aliases are useful for simplifying type hints. For example: 
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def remove_gray_shades ( 
colors: list[tuple[int, int, int]]) -> list[tuplelint, int, int]]: 
pass 


could be made more readable like this: 


Color = tuple[int, int, int] 


def remove_gray_shades (colors: list[Color]) -> list[Color]: 
pass 


See typing and PEP 484, which describe this functionality. 


type hint An annotation that specifies the expected type for a variable, a class attribute, or a function parameter or 
return value. 


Type hints are optional and are not enforced by Python but they are useful to static type analysis tools, and aid 
IDEs with code completion and refactoring. 


Type hints of global variables, class attributes, and functions, but not local variables, can be accessed using 
typing.get_type_hints(). 


See typing and PEP 484, which describe this functionality. 


universal newlines A manner of interpreting text streams in which all of the following are recognized as ending 
a line: the Unix end-of-line convention '\n', the Windows convention '\r\n"', and the old Macintosh 
convention '\r'. See PEP 278 and PEP 3116, as well as bytes. splitlines () for an additional use. 


variable annotation An annotation of a variable or a class attribute. 


When annotating a variable or a class attribute, assignment is optional: 


class C: 
field: 'annotation' 


Variable annotations are usually used for type hints: for example this variable is expected to take int values: 


count: int = 0 


Variable annotation syntax is explained in section annassign. 


See function annotation, PEP 484 and PEP 526, which describe this functionality. Also see annotations-howto 
for best practices on working with annotations. 


virtual environment A cooperatively isolated runtime environment that allows Python users and applications to 
install and upgrade Python distribution packages without interfering with the behaviour of other Python appli- 
cations running on the same system. 


See also venv. 


virtual machine A computer defined entirely in software. Python’s virtual machine executes the bytecode emitted 
by the bytecode compiler. 


Zen of Python Listing of Python design principles and philosophies that are helpful in understanding and using the 
language. The listing can be found by typing “import this” at the interactive prompt. 
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APPENDIX 
B 


ABOUT THESE DOCUMENTS 


These documents are generated from reStructuredText sources by Sphinx, a document processor specifically written 
for the Python documentation. 


Development of the documentation and its toolchain is an entirely volunteer effort, just like Python itself. If you 
want to contribute, please take a look at the reporting-bugs page for information on how to do so. New volunteers 
are always welcome! 


Many thanks go to: 
e Fred L. Drake, Jr., the creator of the original Python documentation toolset and writer of much of the content; 
e the Docutils project for creating reStructuredText and the Docutils suite; 


e Fredrik Lundh for his Alternative Python Reference project from which Sphinx got many good ideas. 


B.1 Contributors to the Python Documentation 


Many people have contributed to the Python language, the Python standard library, and the Python documentation. 
See Misc/ACKS in the Python source distribution for a partial list of contributors. 


It is only with the input and contributions of the Python community that Python has such wonderful documentation 
— Thank You! 
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APPENDIX 
C 


HISTORY AND LICENSE 


C.1 History of the software 


Python was created in the early 1990s by Guido van Rossum at Stichting Mathematisch Centrum (CWI, see https: 
/Iwww.cwi.nl/) in the Netherlands as a successor of a language called ABC. Guido remains Python's principal author, 
although it includes many contributions from others. 


In 1995, Guido continued his work on Python at the Corporation for National Research Initiatives (CNRI, see https: 
//www.cnri.reston.va.us/) in Reston, Virginia where he released several versions of the software. 


In May 2000, Guido and the Python core development team moved to BeOpen.com to form the BeOpen Python- 
Labs team. In October of the same year, the PythonLabs team moved to Digital Creations (now Zope Corporation; 
see https: //www.zope.org/). In 2001, the Python Software Foundation (PSF, see https://www.python.org/psf/) was 
formed, a non-profit organization created specifically to own Python-related Intellectual Property. Zope Corporation 
is a Sponsoring member of the PSF. 


All Python releases are Open Source (see https://opensource.org/ for the Open Source Definition). Historically, most, 
but not all, Python releases have also been GPL-compatible; the table below summarizes the various releases. 


Release Derived from | Year Owner GPL compatible? 
0.9.0 thru 1.2 | n/a 1991-1995 | CWI yes 
1.3 thru 1.5.2 | 1.2 1995-1999 | CNRI yes 
1.6 1.5.2 2000 CNRI no 
2.0 1.6 2000 BeOpen.com | no 
1.6.1 1.6 2001 CNRI no 
2.1 2.0+1.6.1 2001 PSF no 
2.0.1 2.0+1.6.1 2001 PSF yes 
2.1.1 2.14+2.0.1 2001 PSF yes 
2.1.2 2.1.1 2002 PSF yes 
2.1.3 2.1.2 2002 PSF yes 
2.2 and above | 2.1.1 2001-now | PSF yes 


Note: GPL-compatible doesn’t mean that we’re distributing Python under the GPL. All Python licenses, unlike the 
GPL, let you distribute a modified version without making your changes open source. The GPL-compatible licenses 
make it possible to combine Python with other software that is released under the GPL; the others don’t. 


Thanks to the many outside volunteers who have worked under Guido’s direction to make these releases possible. 
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C.2 Terms and conditions for accessing or otherwise using 
Python 


Python software and documentation are licensed under the PSF License Agreement. 


Starting with Python 3.8.6, examples, recipes, and other code in the documentation are dual licensed under the PSF 
License Agreement and the Zero-Clause BSD license. 


Some software incorporated into Python is under different licenses. The licenses are listed with code falling under 
that license. See Licenses and Acknowledgements for Incorporated Software for an incomplete list of these licenses. 


C.2.1 PSF LICENSE AGREEMENT FOR PYTHON 3.11.1 


1. This LICENSE AGREEMENT is between the Python Software Foundation. 
o ("PSF"), and 

the Individual or Organization ("Licensee") accessing and otherwise. 
“using Python 

3.11.1 software in source or binary form and its associated.. 
documentation. 


2. Subject to the terms and conditions of this License Agreement, PSF. 
hereby 

grants Licensee a nonexclusive, royalty-free, world-wide license to. 
“reproduce, 

analyze, test, perform and/or display publicly, prepare derivative. 
o Works, 

distribute, and otherwise use Python 3.11.1 alone or in any derivative 

version, provided, however, that PSF's License Agreement and PSF's 
«notice of 

copyright, i.e., "Copyright O 2001-2022 Python Software Foundation; All., 
Rights 

Reserved" are retained in Python 3.11.1 alone or in any derivative. 
oversion 

prepared by Licensee. 


3. In the event Licensee prepares a derivative work that is based on or 
incorporates Python 3.11.1 or any part thereof, and wants to make the 
derivative work available to others as provided herein, then Licensee 

hereby 
agrees to include in any such work a brief summary of the changes made.. 

“Sto Python 
Salad. 


4. PSF is making Python 3.11.1 available to Licensee on an "AS IS" basis. 
PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY 
EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY. 
SREPRESENTATION OR 
WARRANTY OF ERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOS] 
So THAT THE 
USE OF PYTHON 3.11.1 WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 


El 


OR 


E 


5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 3.11.1 
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A, 

> RESULT OF 

ODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 3.11.1, OR ANY. 

-DERIVATIVE 
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HEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 


6. This License Agreement will automatically terminate upon a materialu 
breach of 
its terms and conditions. 


7. Nothing in this License Agreement shall be deemed to create any. 
relationship 

of agency, partnership, or joint venture between PSF and Licensee. . 
«This License 

Agreement does not grant permission to use PSF trademarks or trade name. 
sin a 


trademark sense to endorse or promote products or services of Licensee, 
Sor any 
third party. 


8. By copying, installing or otherwise using Python 3.11.1, Licensee agrees 
to be bound by the terms and conditions of this License Agreement. 


C.2.2 BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0 


BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1 


1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an office at 
160 Saratoga Avenue, Santa Clara, CA 95051, and the Individual or Organization 


("Licensee") accessing and otherwise using this software in source or binary 
form and its associated documentation ("the Software"). 


2. Subject to the terms and conditions of this BeOpen Python License Agreement, 
BeOpen hereby grants Licensee a non-exclusive, royalty-free, world-wide license 
to reproduce, analyze, test, perform and/or display publicly, prepare derivative 
works, distribute, and otherwise use the Software alone or in any derivative 
version, provided, however, that the BeOpen Python License is retained in the 
Software, alone or in any derivative version prepared by Licensee. 


3. BeOpen is making the Software available to Licensee on an "AS IS" basis. 
BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF 

EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND DISCLAIMS ANY REPRESENTATION OR 
WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE 
USE OF THE SOFTWARE WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 


BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE 

ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF USING, 
MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY DERIVATIVE THEREOF, EVEN IF 
ADVISED OF THE POSSIBILITY THEREOF. 


T 


5. This License Agreement will automatically terminate upon a material breach of 
its terms and conditions. 


6. This License Agreement shall be governed by and interpreted in all respects 
by the law of the State of California, excluding conflict of law provisions. 
Nothing in this License Agreement shall be deemed to create any relationship of 
agency, partnership, or joint venture between BeOpen and Licensee. This License 
Agreement does not grant permission to use BeOpen trademarks or trade names in a 
trademark sense to endorse or promote products or services of Licensee, or any 
third party. As an exception, the "BeOpen Python" logos available at 
http://www.pythonlabs.com/logos.html may be used according to the permissions 
granted on that web page. 


(continues on next page) 
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7. By copying, installing or otherwise using the software, Licensee agrees to b 
bound by the terms and conditions of this License Agreement. 


C.2.3 CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1 


1. This LICENSE AGREEMENT is between the Corporation for National Research 
Initiatives, having an office at 1895 Preston White Drive, Reston, VA 20191 
("CNRI"), and the Individual or Organization ("Licensee") accessing and 
otherwise using Python 1.6.1 software in source or binary form and its 


associated documentation. 


2. Subject to the terms and conditions of this License Agreement, CNRI hereby 
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, 
analyze, test, perform and/or display publicly, prepare derivative works, 
distribute, and otherwise use Python 1.6.1 alone or in any derivative version, 
provided, however, that CNRI's License Agreement and CNRI's notice of copyright, 
i.e., "Copyright © 1995-2001 Corporation for National Research Initiatives; All 
Rights Reserved" are retained in Python 1.6.1 alone or in any derivative version 
prepared by Licensee. Alternately, in lieu of CNRI's License Agreement, 
Licensee may substitute the following text (omitting the quotes): "Python 1.6.1 
is made available subject to the terms and conditions in CNRI's License 
Agreement. This Agreement together with Python 1.6.1 may be located on the 
internet using the following unique, persistent identifier (known as a handle): 
1895.22/1013. This Agreement may also be obtained from a proxy server on the 
internet using the following URL: http://hdl.handle.net/1895.22/1013." 


3. Eb th vent Licens prepares a derivative work that is based on or 
incorporates Python 1.6.1 or any part thereof, and wants to make the derivative 
work available to others as provided herein, then Licens hereby agrees to 
include in any such work a brief summary of the changes made to Python 1.6.1. 


4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS" basis. CNRI 
MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, 
BU OT LIMITATION, CNRI MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY 
OF ERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF 
PYTHON 1.6.1 WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 


= 


5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 1.6.1 FOR 
ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF 
MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1, OR ANY DERIVATIVE 

HEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 


= 


6. This License Agreement will automatically terminate upon a material breach of 


its terms and conditions. 


7. This License Agreement shall be governed by the federal intellectual property 
law of the United States, including without limitation the federal copyright 
law, and, to the extent such U.S. federal law does not apply, by the law of the 
Commonwealth of Virginia, excluding Virginia's conflict of law provisions. 
Notwithstanding the foregoing, with regard to derivative works based on Python 
1.6.1 that incorporate non-separable material that was previously distributed 
under the GNU General Public License (GPL), the law of the Commonwealth of 
Virginia shall govern this License Agreement only as to issues arising under or 
with respect to Paragraphs 4, 5, and 7 of this License Agreement. Nothing in 
this License Agreement shall be deemed to create any relationship of agency, 
partnership, or joint venture between CNRI and Licensee. This License Agreement 
does not grant permission to use CNRI trademarks or trade name in a trademark 
sense to endorse or promote products or services of Licensee, or any third 
party. 


(continues on next page) 
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8. By clicking on the "ACCEPT" button where indicated, or by copying, installing 
or otherwise using Python 1.6.1, Licens agrees to be bound by the terms and 
conditions of this License Agreement. 


C.2.4 CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2 


Copyright © 1991 - 1995, Stichting Mathematisch Centrum Amsterdam, The 
Netherlands. All rights reserved. 


Permission to use, copy, modify, and distribute this software and its 
documentation for any purpose and without fee is hereby granted, provided that 
the above copyright notice appear in all copies and that both that copyright 
notice and this permission notice appear in supporting documentation, and that 
the name of Stichting Mathematisch Centrum or CWI not be used in advertising or 
publicity pertaining to distribution of the software without specific, written 
prior permission. 


STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS 
SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF ERCHANTABILITY AND FITNESS, IN NO 
EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT 
OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, 
DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS 
ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS 
SOFTWARE. 


y 


C.2.5 ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3.11.1 DOCU- 
MENTATION 


Permission to use, copy, modify, and/or distribute this software for any 
purpose with or without fee is hereby granted. 


THE SOFTWARE IS PROVIDED "AS 15" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH 
REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY 

AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, 

INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM 
L 
(0) 
P 


T 


T 


OSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR 
THER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR 
RFORMANCE OF THIS SOFTWARE. 


E 
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C.3 Licenses and Acknowledgements for Incorporated Software 


This section is an incomplete, but growing list of licenses and acknowledgements for third-party software incorporated 
in the Python distribution. 


C.3.1 Mersenne Twister 


The _random module includes code based on a download from http://www.math.sci.hiroshima-u.ac. jp/-m-mat/ 
MT/MT2002/emt19937ar.html. The following are the verbatim comments from the original code: 


A C-program for MT19937, with initialization improved 2002/1/26. 
Coded by Takuji Nishimura and Makoto Matsumoto. 


Before using, initialize the state by using init_genrand (seed) 
or init_by_array(init_key, key_length). 


Copyright (C) 1997 - 2002, Makoto Matsumoto and Takuji Nishimura, 
All rights reserved. 


Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions 
are met: 


1. Redistributions of source code must retain the above copyright 
notice, this list of conditions and the following disclaimer. 


2. Redistributions in binary form must reproduce the above copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 


3. The names of its contributors may not be used to endorse or promote 
products derived from this software without specific prior written 
permission. 


HIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 

"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 

LIMITED TO, THE IMPLIED WARRANTIES OF ERCHANTABILITY AND FITNESS FOR 

A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR 
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 


El 


EXEMPLARY, OR CONSEQUE IAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 


JABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 


Any feedback is very welcome. 
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html 
email: m-mat @ math.sci.hiroshima-u.ac.jp (remove space) 
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C.3.2 Sockets 


The socket module uses the functions, getaddrinfo (), and getnameinfo (), which are coded in separate 
source files from the WIDE Project, https://www.wide.ad. jp/. 


Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. 
All rights reserved. 


Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions 
are met: 

1. Redistributions of source code must r 
notice, this list of conditions and the following disclaimer. 

2. Redistributions in binary form must reproduce the above copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 

3. Neither the name of the project nor the names of its contributors 
may be used to endorse or promote products derived from this software 
without specific prior written permission. 


etain the above copyright 


THIS SOFTWARE IS PROVIDED BY 
ANY EXPRESS OR IMPLIED WARRAN 


HE PROJECT AND CONTRIBUTORS ``AS IS'' AND 
I 
IMPLIED WARRANTIES OF MERCHANTA 
H 
D 


S, INCLUDING, BUT NOT LIMITED TO, THE 
ILITY AND FITNESS FOR A PARTICULAR PURPOSE 
ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE 
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 

OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 
SUCH DAMAGE. 


C.3.3 Asynchronous socket services 


The asynchat and asyncore modules contain the following notice: 


Copyright 1996 by Sam Rushing 
All Rights Reserved 


Permission to use, copy, modify, and distribute this software and 
its documentation for any purpose and without fee is hereby 
granted, provided that the above copyright notice appear in all 
copies and that both that copyright notice and this permission 
notice appear in supporting documentation, and that the name of Sam 
Rushing not be used in advertising or publicity pertaining to 
distribution of the software without specific, written prior 
permission. 


SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, 
INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN 
NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR 
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS 
E, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, 
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN 
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 
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C.3.4 Cookie management 


The http.cookies module contains the following notice: 


Copyright 2000 by Timothy O'Malley <timo@alum.mit.edu> 
All Rights Reserved 


Permission to use, copy, modify, and distribute this software 

and its documentation for any purpose and without fee is hereby 
granted, provided that the above copyright notice appear in all 
copies and that both that copyright notice and this permission 
notice appear in supporting documentation, and that the name of 
Timothy O'Malley not be used in advertising or publicity 

pertaining to distribution of the software without specific, written 
prior permission. 


Timothy O'Malley DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS 
SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY 
AND FITNESS, IN NO EVENT SHALL Timothy O'Malley BE LIABLE FOR 
ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, 
W. 
A 
E 


HETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS 
CTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR 
ERFORMANCE OF THIS SOFTWARE. 


C.3.5 Execution tracing 


The trace module contains the following notice: 


portions copyright 2001, Autonomous Zones Industries, Inc., all rights... 
err... reserved and offered to the public under the terms of the 

Python 2.2 license. 

Author: Zooko O'Whielacronx 

http://zooko.com/ 

mailto:zookoftzooko.com 


Copyright 2000, Mojam Media, Inc., all rights reserved. 
Author: Skip Montanaro 


Copyright 1999, Bioreason, Inc., all rights reserved. 
Author: Andrew Dalke 


Copyright 1995-1997, Automatrix, Inc., all rights reserved. 
Author: Skip Montanaro 


Copyright 1991-1995, Stichting Mathematisch Centrum, all rights reserved. 


Permission to use, copy, modify, and distribute this Python software and 
its associated documentation for any purpose without fee is hereby 
granted, provided that the above copyright notice appears in all copies, 
and that both that copyright notice and this permission notice appear in 
supporting documentation, and that the name of neither Automatrix, 
Bioreason or Mojam Media be used in advertising or publicity pertaining to 
distribution of the software without specific, written prior permission. 
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C.3.6 UUencode and UUdecode functions 


The uu module contains the following notice: 


Copyright 1994 by Lance Ellinghouse 
Cathedral City, California Republic, United States of America. 

All Rights Reserved 
Permission to use, copy, modify, and distribute this software and its 
documentation for any purpose and without fee is hereby granted, 
provided that the above copyright notice appear in all copies and that 
both that copyright notice and this permission notice appear in 
supporting documentation, and that the name of Lance Ellinghouse 
not be used in advertising or publicity pertaining to distribution 
of the software without specific, written prior permission. 
ANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO 
HIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND 
FITNESS, IN NO EVENT SHALL LANCE ELLINGHOUSE CENTRUM BE LIABLE 
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT 
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF HIS SOFTWARE. 


Tj 


Modified by Jack Jansen, CWI, July 1995: 
- Use binascii module to do the actual line-by-line conversion 
between ascii and binary. This results in a 1000-fold speedup. The C 
version is still 5 times faster, though. 
- Arguments more compliant with Python standard 


C.3.7 XML Remote Procedure Calls 


The xm1rpc. client module contains the following notice: 


The XML-RPC client interface is 


Copyright (c) 1999-2002 by Secret Labs AB 
Copyright (c) 1999-2002 by Fredrik Lundh 


By obtaining, using, and/or copying this software and/or its 
associated documentation, you agree that you have read, understood, 
and will comply with the following terms and conditions: 


Permission to use, copy, modify, and distribute this software and 
its associated documentation for any purpose and without fee is 
hereby granted, provided that the above copyright notice appears in 
all copies, and that both that copyright notice and this permission 
notice appear in supporting documentation, and that the name of 
Secret Labs AB or the author not be used in advertising or publicity 
pertaining to distribution of the software without specific, written 
prior permission. 


ECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD 

O THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- 
BILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR 
E LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY 
AMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, 
ETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS 
ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANC 
OF THIS SOFTWARE. 


SuvuwryHn 


oo 


E 
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C.3.8 test_epoll 


The test_epo11 module contains the following notice: 


Copyright (c) 2001-2006 Twisted Matrix Laboratories. 


Permission is hereby granted, free of charge, to any person obtaining 
a copy of this software and associated documentation files (the 
"Software"), to deal in the Software without restriction, including 
without limitation the rights to use, copy, modify, merge, publish, 
distribute, sublicense, and/or sell copies of the Software, and to 
permit persons to whom the Software is furnished to do so, subject to 
the following conditions: 

The above copyright notice and this permission notice shall be 
included in all copies or substantial portions of the Software. 


HE SOFTWARE IS PROVIDED "AS 15", WITHOUT WARRANTY OF ANY KIND, 
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 
ERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 
TABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 


C.3.9 Select kqueue 


The select module contains the following notice for the kqueue interface: 


Copyright (c) 2000 Doug White, 2006 James Knight, 2007 Christian Heimes 
All rights reserved. 


Redistribution and use in source and binary forms, with or without 

modification, are permitted provided that the following conditions 

are met: 

1. Redistributions of source code must retain the above copyright 
notice, this list of conditions and the following disclaimer. 

2. Redistributions in binary form must reproduce the above copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 


THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
H 
D 


ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 

OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, W ER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 
SUCH DAMAGE. 


am 
T 
I 


T 
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C.3.10 SipHash24 


The file Python /pyhash. c contains Marek Majkowski’ implementation of Dan Bernstein's SipHash24 algorithm. 
It contains the following note: 


<MIT License> 
Copyright (c) 2013 Marek Majkowski <marek@popcount.org> 


Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights 
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
copies of the Software, and to permit persons to whom the Software is 
furnished to do so, subject to the following conditions: 


The above copyright notice and this permission notice shall be included in 
all copies or substantial portions of the Software. 
</MIT License> 


Original location: 
https://github.com/majek/csiphash/ 


Solution inspired by code from: 
Samuel Neves (supercop/crypto_auth/siphash24/little) 
djb (supercop/crypto_auth/siphash24/little2) 
Jean-Philippe Aumasson (https://131002.net/siphash/siphash24.c) 


C.3.11 strtod and dtoa 


The file Python/dtoa.c, which supplies C functions dtoa and strtod for conversion of C doubles to and from 
strings, is derived from the file of the same name by David M. Gay, currently available from https://web.archive.org/ 
web/202205 17033456/http://www.netlib.org/fp/dtoa.c. The original file, as retrieved on March 16, 2009, contains 
the following copyright and licensing notice: 


[KKK I KK ARK RAR A RA RRA RARA RARA A I KK RAR 


The author of this software is David M. Gay. 
Copyright (c) 1991, 2000, 2001 by Lucent Technologies. 


* 
* 
* 
* 
* Permission to use, copy, modify, and distribute this software for any 

* purpose without fee is hereby granted, provided that this entire notice 
* is included in all copies of any software which is or includes a copy 

* or modification of this software and in all copies of the supporting 

* documentation for such software. 

* 

* 

* 

* 

* 

* 

* 


THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMP 
WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR LUCENT MAKES ANY 
REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY 
Or THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. 


IED 


A 
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C.3.12 OpenSSL 


The modules hashlib, posix, ssl, crypt use the OpenSSL library for added performance if made available 
by the operating system. Additionally, the Windows and macOS installers for Python may include a copy of the 
OpenSSL libraries, so we include a copy of the OpenSSL license here: 


LICENSE ISSUES 
The OpenSSL toolkit stays under a dual license, i.e. both the conditions of 
the OpenSSL License and the original SSLeay license apply to the toolkit. 


See below 


Open Source licenses. 


please con 


for the actual license texts. Actually both licenses are BSD-styl 
In case of any license issues related to OpenSSL 
tact openssl-core@openssl.org. 


OpenSSL License 


Copyrigh 


Redistribution and use in source and binary forms, 
modifica 
are met: 


Redis 
notice, 


Redis 
notice, 
the documentation and/or other materials provided 
distribution. 


The names 
endorse or promote products derived from this software without 
prior written permission. 


t (c) 1998-2008 The OpenSSL Project. All rights reserved. 


with or without 


tion, are permitted provided that the following conditions 


tributions of source code must 
this list of conditions and 


retain the above copyright 
the following disclaimer. 


tributions in binary form must 
this list of conditions and 


reproduce th 
the following 


above copyright 
disclaimer in 
with the 


All advertising materials mentioning features or use of this 
software must display the following acknowledgment: 

"This product includes software developed by the OpenSSL Project 
for use in the OpenSSL Toolkit. 


(http: //www.openssl.org/)" 
"OpenSSL Toolkit" and "OpenSSL Project" must not be used to 


For written permission, please contact 


Op 


Redistributions of any 
acknowledgment: 

"This product includes 
for use in the OpenSSL 


THIS SOFTWAR 


nssl-core@openssl.org. 


Products derived from this software may not be called "OpenSSL" 
nor may 
permission of the OpenSSL Project. 


"OpenSSL" appear in their names without prior written 


form whatsoever must retain the following 


software developed by the OpenSSL Project 
Toolkit (http://www.openssl.org/)" 


E E 


Bh 


IS PROVIDED BY OpenSSL PROJECT ~*AS IS'' AND ANY 


EXPR 


ESS 


E 


ED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THI 


IMPLII 


ED WA 
PURPOS 


E, 


RRANTIES 
DISCLA 


OF MERCHAN 
IMED. IN NO 


ABILITY AND FIT 
EVENT SHALL THI 


SS FOR A PARTICULAR 
OpenSSL PROJECT OR 


Ra] 


AR 


ITS CO 


TRIBUTORS B 


LIAB FOR 


ANY DIRECT, INDIRECT, INCIDENTAL, 


SP 


EX 


T 


ECIAL, 
NOT LI 
LOSS OF USE 


PLARY, 
TO, PROC 


OR CONSEQU 
UREMENT OF 
DATA, OR PROFITS; 


ENTIAL DAMAGES (INCLUDING, BUT 
SUBSTITUTE GOODS OR SERVICES; 
OR BUSINESS INTERRUPTION) 


ITED 


By 


HOW 


EV 


R CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 


+ F + FF FF F FF F F F FF F F F F F F F F F F F F F F F F F F F FF F F F F F F F AC F F F KF F 


STRICT LIABILITY, 


OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 


F 
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+ AX A AX A F F ++ 


ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED 
OF THE POSSIBILITY OF SUCH DAMAGE. 


This product includes cryptographic software written by Eric Young 
(eay@cryptsoft.com). This product includes software written by Tim 
Hudson (tjh@cryptsoft.com). 


Original SSLeay License 


+ + + FF F F F FF F F FF F F F F F F F F F F F F FF F F FF F FF F F FF F F F F F A F F F F F F F KF F 


Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) 
All rights reserved. 


This package is an SSL implementation written 
by Eric Young (eay@cryptsoft.com). 
The implementation was written so as to conform with Netscapes SSL. 


This library is free for commercial and non-commercial use as long as 
the following conditions are aheared to. The following conditions 
apply to all code found in this distribution, be it the RC4, RSA, 
lhash, DES, etc., code; not just the SSL code. The SSL documentation 
included with this distribution is covered by the same copyright terms 
except that the holder is Tim Hudson (tjh@cryptsoft.com). 


Copyright remains Eric Young's, and as such any Copyright notices in 

the code are not to be removed. 

If this package is used in a product, Eric Young should be given attribution 
as the author of the parts of the library used. 

This can be in the form of a textual message at program startup or 

in documentation (online or textual) provided with the package. 


Redistribution and use in source and binary forms, with or without 

modification, are permitted provided that the following conditions 

are met: 

1. Redistributions of source code must retain the copyright 
notice, this list of conditions and the following disclaimer. 

2. Redistributions in binary form must reproduce the above copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 

3. All advertising materials mentioning features or use of this software 
must display the following acknowledgement: 

"This product includes cryptographic software written by 

Eric Young (eay@cryptsoft.com)" 

The word 'cryptographic' can be left out if the rouines from the library 

being used are not cryptographic related :-). 

I 

t 


f you include any Windows specific code (or a derivative thereof) from 
he apps directory (application code) you must include an acknowledgement : 
"This product includes software written by Tim Hudson (tjh@cryptsoft.com) " 


THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND 

ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 

OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 


T 
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* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 

* SUCH DAMAGE. 

* 

* The licence and distribution terms for any publically available version or 
* derivative of this code cannot be changed. i.e. this code cannot simply be 
* copied and put under another distribution licence 

* [including the GNU Public Licence. ] 

*/ 


C.3.13 expat 


The pyexpat extension is built using an included copy of the expat sources unless the build is configured 
with-system-expat: 


Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd 
and Clark Cooper 


Permission is hereby granted, free of charge, to any person obtaining 
a copy of this software and associated documentation files (the 
"Software"), to deal in the Software without restriction, including 
without limitation the rights to use, copy, modify, merge, publish, 
distribute, sublicense, and/or sell copies of the Software, and to 
permit persons to whom the Software is furnished to do so, subject to 
the following conditions: 

The above copyright notice and this permission notice shall be included 
in all copies or substantial portions of the Software. 


HE SOFTWARE IS PROVIDED "AS 15", WITHOUT WARRANTY OF ANY KIND, 
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 
ERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 


C.3.14 libffi 


The _ctypes extension is built using an included copy of the libffi sources unless the build is configured 
with-system-libffi: 


Copyright (c) 1996-2008 Red Hat, Inc and others. 


Permission is hereby granted, free of charge, to any person obtaining 
a copy of this software and associated documentation files (the 
"*Software''), to deal in the Software without restriction, including 
without limitation the rights to use, copy, modify, merge, publish, 
distribute, sublicense, and/or sell copies of the Software, and to 
permit persons to whom the Software is furnished to do so, subject to 
the following conditions: 


The above copyright notice and this permission notice shall be included 
in all copies or substantial portions of the Software. 


THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, 
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 
RCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 


E 
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NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
DEALINGS IN THE SOFTWARE. 


C.3.15 zlib 


The zlib extension is built using an included copy of the zlib sources if the zlib version found on the system is too 
old to be used for the build: 


Copyright (C) 1995-2011 Jean-loup Gailly and Mark Adler 


This software is provided 'as-is', without any express or implied 
warranty. In no event will the authors be held liable for any damages 
arising from the use of this software. 


Permission is granted to anyone to use this software for any purpose, 
including commercial applications, and to alter it and redistribute it 
freely, subject to the following restrictions: 


1. The origin of this software must not be misrepresented; you must not 
claim that you wrote the original software. If you use this software 
in a product, an acknowledgment in the product documentation would be 
appreciated but is not required. 


2. Altered source versions must be plainly marked as such, and must not be 
misrepresented as being the original software. 


3. This notice may not be removed or altered from any source distribution. 


Jean-loup Gailly Mark Adler 
jloup@gzip.org madler@alumni.caltech.edu 


C.3.16 cfuhash 


The implementation of the hash table used by the tracemal loc is based on the cfuhash project: 


Copyright (c) 2005 Don Owens 
All rights reserved. 


This code is released under the BSD license: 


Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions 
are met: 


* Redistributions of source code must retain the above copyright 
notice, this list of conditions and the following disclaimer. 


* Redistributions in binary form must reproduce the abov 
copyright notice, this list of conditions and the following 
disclaimer in the documentation and/or other materials provided 
with the distribution. 


* Neither the name of the author nor the names of its 


contributors may be used to endorse or promote products derived 
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from this software without specific prior written permission. 


HIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED 
OF THE POSSIBILITY OF SUCH DAMAGE. 


C.3.17 libmpdec 


The _decimal module is built using an included copy of the libmpdec library unless the build is configured 
with-system-libmpdec: 


Copyright (c) 2008-2020 Stefan Krah. All rights reserved. 


Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions 
are met: 


1. Redistributions of source code must retain the above copyright 
notice, this list of conditions and the following disclaimer. 


2. Redistributions in binary form must reproduce the above copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 


THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS" AND 
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
H 
D 


ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
DAMAGES (INCLUDING, BUT NO ,IMITED TO, PROCUREME OF SUBSTITUTE GOODS 

OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHE ER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 
OUT OF THE USE OF THIS SOFTWARE, EVE IF ADVISED OF THE POSSIBILITY OF 
SUCH DAMAGE. 


T 


C.3.18 W3C C14N test suite 


The C14N 2.0 test suite in the test package (Lib/test /xmltestdata/c14n-20/) was retrieved from the 
W3C website at https://www.w3.org/TR/xml-c14n2-testcases/ and is distributed under the 3-clause BSD license: 


Copyright (c) 2013 W3C(R) (MIT, ERCIM, Keio, Beihang), 
All Rights Reserved. 


Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions 
are met: 
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* Redistributions of works must retain the original copyright notice, 
this list of conditions and the following disclaimer. 

* Redistributions in binary form must reproduce the original copyright 
notice, this list of conditions and the following disclaimer in the 
documentation and/or other materials provided with the distribution. 

* Neither the name of the W3C nor the names of its contributors may be 
used to endorse or promote products derived from this work without 
specific prior written permission. 


HIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 
HEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 


| 


C.3.19 Audioop 


The audioop module uses the code base in g771.c file of the SoX project: 


Programming the AdLib/Sound Blaster 

FM Music Chips 

Version 2.0 (24 Feb 1992) 

Copyright (c) 1991, 1992 by Jeffrey S. Lee 

jlee@smylex.uucp 

Warranty and Copyright Policy 

This document is provided on an "as-is" basis, and its author makes 
no warranty or representation, express or implied, with respect to 
its quality performance or fitness for a particular purpose. In no 
event will the author of this document be liable for direct, indirect, 
special, incidental, or consequential damages arising out of the use 
or inability to use the information contained within. Use of this 
document is at your own risk. 
This file may be used and copied freely so long as the applicable 
copyright notices are retained, and no modifications are made to the 
text of the document. No money shall be charged for its distribution 
beyond reasonable shipping, handling and duplication costs, nor shall 
proprietary changes be made to this document so that it cannot be 
distributed freely. This document may not be included in published 
material or commercial packages without the written consent of its 
author. 
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APPENDIX 
D 


COPYRIGHT 


Python and this documentation is: 

Copyright O 2001-2022 Python Software Foundation. All rights reserved. 

Copyright O 2000 BeOpen.com. All rights reserved. 

Copyright O 1995-2000 Corporation for National Research Initiatives. All rights reserved. 
Copyright O 1991-1995 Stichting Mathematisch Centrum. All rights reserved. 


See History and License for complete license and permissions information. 
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